[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:46:16 dist-test-slave-f0tn krb5kdc[7625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:46:16 dist-test-slave-f0tn krb5kdc[7625](info): set up 2 sockets
Aug 11 12:46:16 dist-test-slave-f0tn krb5kdc[7625](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 12:46:17.191401 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.003s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:46:17 dist-test-slave-f0tn krb5kdc[7625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:46:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:46:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:46:17.377254 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:40735
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:43485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:40735
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:17.758610 7643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:17.759006 7643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:17.759135 7643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:17.767167 7643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:46:17.767290 7643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:17.767354 7643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:17.767421 7643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:46:17.767498 7643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:46:17.778281 7643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:43485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:40735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:40735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:17.780061 7643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:17.782110 7643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:17.797294 7654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:17.797833 7656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:17.798115 7653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:17.799754 7643 server_base.cc:1034] running on GCE node
I20260811 12:46:17.800638 7643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:17.802394 7643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:17.803613 7643 hybrid_clock.cc:648] HybridClock initialized: now 1786452377803567 us; error 69 us; skew 500 ppm
I20260811 12:46:17.804026 7643 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:17 dist-test-slave-f0tn krb5kdc[7625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452377, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:17.810374 7643 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:46:17.812060 7643 webserver.cc:460] Webserver started at http://127.7.111.126:44759/ using document root <none> and password file <none>
I20260811 12:46:17.812918 7643 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:17.813046 7643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:17.813509 7643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:17.817179 7643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "38715379d5e046c08b97b73d66dcaddf"
format_stamp: "Formatted at 2026-08-11 12:46:17 on dist-test-slave-f0tn"
server_key: "eb03d974e8d761daa94d8ddd3d42885d"
server_key_iv: "57c9dac0ee785fb5ddf71b03e13b8331"
server_key_version: "encryptionkey@0"
I20260811 12:46:17.818087 7643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "38715379d5e046c08b97b73d66dcaddf"
format_stamp: "Formatted at 2026-08-11 12:46:17 on dist-test-slave-f0tn"
server_key: "eb03d974e8d761daa94d8ddd3d42885d"
server_key_iv: "57c9dac0ee785fb5ddf71b03e13b8331"
server_key_version: "encryptionkey@0"
I20260811 12:46:17.824213 7643 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.002s
I20260811 12:46:17.827931 7668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:17.829362 7643 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:46:17.829563 7643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "38715379d5e046c08b97b73d66dcaddf"
format_stamp: "Formatted at 2026-08-11 12:46:17 on dist-test-slave-f0tn"
server_key: "eb03d974e8d761daa94d8ddd3d42885d"
server_key_iv: "57c9dac0ee785fb5ddf71b03e13b8331"
server_key_version: "encryptionkey@0"
I20260811 12:46:17.829799 7643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:17.849007 7643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:17.853407 7643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:17.853681 7643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:17.876605 7643 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:40735
I20260811 12:46:17.876662 7750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:40735 every 8 connection(s)
I20260811 12:46:17.878271 7643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:46:17.883039 7751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:17.885396 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 7643
I20260811 12:46:17.885764 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:46:17.888334 7613 external_mini_cluster.cc:1428] Setting key c129f35ec2fd4bf08367a7f71768a277
I20260811 12:46:17.897663 7751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38715379d5e046c08b97b73d66dcaddf: Bootstrap starting.
I20260811 12:46:17.900954 7751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38715379d5e046c08b97b73d66dcaddf: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:17.902115 7751 log.cc:824] T 00000000000000000000000000000000 P 38715379d5e046c08b97b73d66dcaddf: Log is configured to *not* fsync() on all Append() calls
I20260811 12:46:17.905797 7751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38715379d5e046c08b97b73d66dcaddf: No bootstrap required, opened a new log
I20260811 12:46:17.915689 7751 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38715379d5e046c08b97b73d66dcaddf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38715379d5e046c08b97b73d66dcaddf" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40735 } }
I20260811 12:46:17.916061 7751 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38715379d5e046c08b97b73d66dcaddf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:17.916189 7751 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38715379d5e046c08b97b73d66dcaddf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38715379d5e046c08b97b73d66dcaddf, State: Initialized, Role: FOLLOWER
I20260811 12:46:17.916903 7751 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38715379d5e046c08b97b73d66dcaddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38715379d5e046c08b97b73d66dcaddf" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40735 } }
I20260811 12:46:17.917289 7751 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38715379d5e046c08b97b73d66dcaddf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:46:17.917471 7751 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38715379d5e046c08b97b73d66dcaddf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:46:17.917656 7751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38715379d5e046c08b97b73d66dcaddf [term 0 FOLLOWER]: Advancing to term 1
Aug 11 12:46:17 dist-test-slave-f0tn krb5kdc[7625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:17.920822 7751 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38715379d5e046c08b97b73d66dcaddf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38715379d5e046c08b97b73d66dcaddf" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40735 } }
I20260811 12:46:17.921484 7751 leader_election.cc:302] T 00000000000000000000000000000000 P 38715379d5e046c08b97b73d66dcaddf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38715379d5e046c08b97b73d66dcaddf; no voters:
I20260811 12:46:17.922274 7751 leader_election.cc:288] T 00000000000000000000000000000000 P 38715379d5e046c08b97b73d66dcaddf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:46:17.922439 7756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38715379d5e046c08b97b73d66dcaddf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:46:17.924494 7756 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38715379d5e046c08b97b73d66dcaddf [term 1 LEADER]: Becoming Leader. State: Replica: 38715379d5e046c08b97b73d66dcaddf, State: Running, Role: LEADER
I20260811 12:46:17.925083 7751 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38715379d5e046c08b97b73d66dcaddf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:46:17.925271 7756 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38715379d5e046c08b97b73d66dcaddf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38715379d5e046c08b97b73d66dcaddf" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40735 } }
I20260811 12:46:17.928934 7760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38715379d5e046c08b97b73d66dcaddf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38715379d5e046c08b97b73d66dcaddf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38715379d5e046c08b97b73d66dcaddf" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40735 } } }
I20260811 12:46:17.929466 7760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38715379d5e046c08b97b73d66dcaddf [sys.catalog]: This master's current role is: LEADER
I20260811 12:46:17.934036 7762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38715379d5e046c08b97b73d66dcaddf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38715379d5e046c08b97b73d66dcaddf. Latest consensus state: current_term: 1 leader_uuid: "38715379d5e046c08b97b73d66dcaddf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38715379d5e046c08b97b73d66dcaddf" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40735 } } }
I20260811 12:46:17.935058 7753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:17.893572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51456 (local address 127.7.111.126:40735)
0811 12:46:17.894869 (+ 1297us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:17.894884 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:17.896247 (+ 1363us) server_negotiation.cc:407] Connection header received
0811 12:46:17.906258 (+ 10011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:17.906291 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:17.906751 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:17.907157 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:17.909905 (+ 2748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:17.912025 (+ 2120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:17.913702 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:17.914866 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:17.915081 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:17.922125 (+ 7044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:17.922134 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:17.922148 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:17.922195 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:17.926278 (+ 4083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:17.927214 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:17.927224 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:17.927232 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:17.927491 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:17.928155 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:17.928162 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:17.928166 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:17.928983 (+ 817us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:17.929503 (+ 520us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:17.932422 (+ 2919us) server_negotiation.cc:299] Negotiation successful
0811 12:46:17.932900 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":961,"thread_start_us":573,"threads_started":1}
I20260811 12:46:17.935209 7762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38715379d5e046c08b97b73d66dcaddf [sys.catalog]: This master's current role is: LEADER
I20260811 12:46:17.941936 7767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:46:17.948263 7767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:46:17.958235 7767 catalog_manager.cc:1347] Generated new cluster ID: e335ae2f40b94e78aa86544214935bcc
I20260811 12:46:17.958340 7767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:46:17.966892 7767 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:46:17.968058 7767 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:46:17.976639 7767 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38715379d5e046c08b97b73d66dcaddf: Generated new TSK 0
I20260811 12:46:17.977697 7767 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.111.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
I20260811 12:46:18.033584 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40735
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:43485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:18.408675 7793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:18.408993 7793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:18.409124 7793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:18.416656 7793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:18.416785 7793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:18.416924 7793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:46:18.427225 7793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:43485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40735
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:18.428459 7793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:18.429900 7793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:18.443895 7801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:18.443861 7806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:18.443949 7802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:18.450218 7793 server_base.cc:1034] running on GCE node
I20260811 12:46:18.450840 7793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:18.451860 7793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:18.453060 7793 hybrid_clock.cc:648] HybridClock initialized: now 1786452378452998 us; error 54 us; skew 500 ppm
I20260811 12:46:18.453411 7793 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:18 dist-test-slave-f0tn krb5kdc[7625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452378, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:18.456820 7793 init.cc:377] Logged in from keytab as kudu/127.7.111.65@KRBTEST.COM (short username kudu)
I20260811 12:46:18.458468 7793 webserver.cc:460] Webserver started at http://127.7.111.65:44453/ using document root <none> and password file <none>
I20260811 12:46:18.459196 7793 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:18.459312 7793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:18.459623 7793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:18.462864 7793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "b07a116a3da64e2b9dcec9b3b7b50eb7"
format_stamp: "Formatted at 2026-08-11 12:46:18 on dist-test-slave-f0tn"
server_key: "6f2fd5bd8203e2089be129e5ab364302"
server_key_iv: "edcc5a3e12fb031d9e01c6a0780f6313"
server_key_version: "encryptionkey@0"
I20260811 12:46:18.463877 7793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "b07a116a3da64e2b9dcec9b3b7b50eb7"
format_stamp: "Formatted at 2026-08-11 12:46:18 on dist-test-slave-f0tn"
server_key: "6f2fd5bd8203e2089be129e5ab364302"
server_key_iv: "edcc5a3e12fb031d9e01c6a0780f6313"
server_key_version: "encryptionkey@0"
I20260811 12:46:18.469758 7793 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:46:18.473538 7822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:18.474906 7793 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:46:18.475065 7793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "b07a116a3da64e2b9dcec9b3b7b50eb7"
format_stamp: "Formatted at 2026-08-11 12:46:18 on dist-test-slave-f0tn"
server_key: "6f2fd5bd8203e2089be129e5ab364302"
server_key_iv: "edcc5a3e12fb031d9e01c6a0780f6313"
server_key_version: "encryptionkey@0"
I20260811 12:46:18.475252 7793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:18.506042 7793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:18.510015 7793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:18.510290 7793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:18.511335 7793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:46:18.513165 7793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:18.513260 7793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:18.513451 7793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:18.513533 7793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:18.555343 7793 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:44613
I20260811 12:46:18.555394 8014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:44613 every 8 connection(s)
I20260811 12:46:18.556826 7793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:46:18.564585 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 7793
I20260811 12:46:18.564811 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:46:18.565477 7613 external_mini_cluster.cc:1428] Setting key 4505ff97a829c822b1cb03cf811c6928
Aug 11 12:46:18 dist-test-slave-f0tn krb5kdc[7625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452378, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:18.578473 8018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:18.559677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:38621 (local address 127.7.111.126:40735)
0811 12:46:18.560236 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:18.560242 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:18.561184 (+ 942us) server_negotiation.cc:407] Connection header received
0811 12:46:18.563157 (+ 1973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:18.563162 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:18.563236 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:18.563332 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:18.564768 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:18.565732 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:18.567837 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:18.568360 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:18.568474 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:18.572527 (+ 4053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:18.572532 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:18.572535 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:18.572567 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:18.575263 (+ 2696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:18.576169 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:18.576172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:18.576174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:18.576316 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:18.576785 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:18.576788 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:18.576790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:18.577026 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:18.577130 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:18.578006 (+ 876us) server_negotiation.cc:299] Negotiation successful
0811 12:46:18.578165 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":281,"threads_started":1}
I20260811 12:46:18.579977 8022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:18.560247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40735 (local address 127.7.111.65:38621)
0811 12:46:18.561016 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:18.561055 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:18.562897 (+ 1842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:18.563524 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:18.563532 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:18.564030 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:18.564571 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:18.564588 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:18.565886 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:18.565891 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:18.567705 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:18.567718 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:18.568580 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:18.568588 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:18.568854 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:18.569536 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:18.569557 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:18.572314 (+ 2757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:18.575416 (+ 3102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:18.575426 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:18.575444 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:18.576063 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:18.576418 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:18.576423 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:18.576427 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:18.576692 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:18.577145 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:18.577153 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:18.577602 (+ 449us) client_negotiation.cc:769] Sending connection context
0811 12:46:18.577938 (+ 336us) client_negotiation.cc:240] Negotiation successful
0811 12:46:18.578518 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":313,"threads_started":1}
I20260811 12:46:18.581553 8016 heartbeater.cc:347] Connected to a master server at 127.7.111.126:40735
I20260811 12:46:18.581905 8016 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:18.582899 8016 heartbeater.cc:511] Master 127.7.111.126:40735 requested a full tablet report, sending...
I20260811 12:46:18.585667 7696 ts_manager.cc:194] Registered new tserver with Master: b07a116a3da64e2b9dcec9b3b7b50eb7 (127.7.111.65:44613)
I20260811 12:46:18.587411 7696 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:38621
WARNING: no policy specified for kudu/127.7.111.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
I20260811 12:46:18.622709 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40735
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:43485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:19.004352 8028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:19.004650 8028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:19.004796 8028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:19.012390 8028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:19.012511 8028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:19.012672 8028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:46:19.022961 8028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:43485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40735
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:19.024354 8028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:19.025779 8028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:19.040030 8034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:19.040238 8040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:19.040534 8037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:19.041568 8028 server_base.cc:1034] running on GCE node
I20260811 12:46:19.042146 8028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:19.043074 8028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:19.044275 8028 hybrid_clock.cc:648] HybridClock initialized: now 1786452379044233 us; error 42 us; skew 500 ppm
I20260811 12:46:19.044600 8028 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:19 dist-test-slave-f0tn krb5kdc[7625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452379, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:19.047966 8028 init.cc:377] Logged in from keytab as kudu/127.7.111.66@KRBTEST.COM (short username kudu)
I20260811 12:46:19.049477 8028 webserver.cc:460] Webserver started at http://127.7.111.66:39365/ using document root <none> and password file <none>
I20260811 12:46:19.050195 8028 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:19.050348 8028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:19.050727 8028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:19.053925 8028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "4560624a29a844a0b969e3726256c2fc"
format_stamp: "Formatted at 2026-08-11 12:46:19 on dist-test-slave-f0tn"
server_key: "258e7c34141c97b18ba0a1492cb023a9"
server_key_iv: "33bdf936b2facb5e4eaae0f7aef708d6"
server_key_version: "encryptionkey@0"
I20260811 12:46:19.054702 8028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "4560624a29a844a0b969e3726256c2fc"
format_stamp: "Formatted at 2026-08-11 12:46:19 on dist-test-slave-f0tn"
server_key: "258e7c34141c97b18ba0a1492cb023a9"
server_key_iv: "33bdf936b2facb5e4eaae0f7aef708d6"
server_key_version: "encryptionkey@0"
I20260811 12:46:19.060107 8028 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:46:19.064096 8053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:19.065469 8028 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:46:19.065651 8028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "4560624a29a844a0b969e3726256c2fc"
format_stamp: "Formatted at 2026-08-11 12:46:19 on dist-test-slave-f0tn"
server_key: "258e7c34141c97b18ba0a1492cb023a9"
server_key_iv: "33bdf936b2facb5e4eaae0f7aef708d6"
server_key_version: "encryptionkey@0"
I20260811 12:46:19.065852 8028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:19.086741 8028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:19.090939 8028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:19.091212 8028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:19.092149 8028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:46:19.093586 8028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:19.093677 8028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:19.093811 8028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:19.093876 8028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:19.136802 8028 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:34287
I20260811 12:46:19.136842 8238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:34287 every 8 connection(s)
I20260811 12:46:19.138235 8028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:46:19.139307 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 8028
I20260811 12:46:19.139556 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:46:19.140158 7613 external_mini_cluster.cc:1428] Setting key 0fa4561e3e36bd9ba18a8b63069a0983
Aug 11 12:46:19 dist-test-slave-f0tn krb5kdc[7625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452379, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:19.164994 8246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:19.141501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:47561 (local address 127.7.111.126:40735)
0811 12:46:19.141958 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:19.141962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:19.143852 (+ 1890us) server_negotiation.cc:407] Connection header received
0811 12:46:19.146917 (+ 3065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:19.146922 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:19.146992 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:19.147079 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:19.149442 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:19.150326 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:19.152864 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:19.153652 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:19.153752 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:19.158290 (+ 4538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:19.158297 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:19.158301 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:19.158345 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:19.161848 (+ 3503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:19.162546 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:19.162551 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:19.162555 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:19.162725 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:19.163122 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:19.163127 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:19.163130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:19.163488 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:19.163635 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:19.164334 (+ 699us) server_negotiation.cc:299] Negotiation successful
0811 12:46:19.164554 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":258,"threads_started":1}
I20260811 12:46:19.166024 8247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:19.142468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40735 (local address 127.7.111.66:47561)
0811 12:46:19.143662 (+ 1194us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:19.143723 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:19.146541 (+ 2818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:19.147397 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:19.147411 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:19.148307 (+ 896us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:19.149209 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:19.149238 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:19.150507 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:19.150513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:19.152658 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:19.152675 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:19.153887 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:19.153894 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:19.154189 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:19.154977 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:19.155015 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:19.158017 (+ 3002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:19.161979 (+ 3962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:19.161986 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:19.162000 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:19.162428 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:19.162821 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:19.162824 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:19.162826 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:19.163008 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:19.163602 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:19.163608 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:19.163938 (+ 330us) client_negotiation.cc:769] Sending connection context
0811 12:46:19.164217 (+ 279us) client_negotiation.cc:240] Negotiation successful
0811 12:46:19.164639 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"thread_start_us":510,"threads_started":1}
I20260811 12:46:19.167557 8239 heartbeater.cc:347] Connected to a master server at 127.7.111.126:40735
I20260811 12:46:19.167984 8239 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:19.169003 8239 heartbeater.cc:511] Master 127.7.111.126:40735 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.111.67@KRBTEST.COM; defaulting to no policy
I20260811 12:46:19.170845 7696 ts_manager.cc:194] Registered new tserver with Master: 4560624a29a844a0b969e3726256c2fc (127.7.111.66:34287)
I20260811 12:46:19.171689 7696 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:47561
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
I20260811 12:46:19.193796 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40735
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:43485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:19.572322 8252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:19.572603 8252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:19.572777 8252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:19.580574 8252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:19.580677 8252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:19.580825 8252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:46:19.590333 8016 heartbeater.cc:503] Master 127.7.111.126:40735 was elected leader, sending a full tablet report...
I20260811 12:46:19.591215 8252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:43485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40735
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:19.592561 8252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:19.593895 8252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:19.607105 8258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:19.607168 8264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:19.607329 8260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:19.608880 8252 server_base.cc:1034] running on GCE node
I20260811 12:46:19.609552 8252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:19.610526 8252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:19.611763 8252 hybrid_clock.cc:648] HybridClock initialized: now 1786452379611687 us; error 52 us; skew 500 ppm
I20260811 12:46:19.612102 8252 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:19 dist-test-slave-f0tn krb5kdc[7625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452379, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:19.615669 8252 init.cc:377] Logged in from keytab as kudu/127.7.111.67@KRBTEST.COM (short username kudu)
I20260811 12:46:19.617122 8252 webserver.cc:460] Webserver started at http://127.7.111.67:44579/ using document root <none> and password file <none>
I20260811 12:46:19.617831 8252 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:19.617950 8252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:19.618323 8252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:19.621225 8252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "5b53dc0348f9496fb6f844943934399f"
format_stamp: "Formatted at 2026-08-11 12:46:19 on dist-test-slave-f0tn"
server_key: "3fec19a340d063fa915032d6322ca1c1"
server_key_iv: "280628aa09e2c6ae077540631b211e5c"
server_key_version: "encryptionkey@0"
I20260811 12:46:19.622017 8252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "5b53dc0348f9496fb6f844943934399f"
format_stamp: "Formatted at 2026-08-11 12:46:19 on dist-test-slave-f0tn"
server_key: "3fec19a340d063fa915032d6322ca1c1"
server_key_iv: "280628aa09e2c6ae077540631b211e5c"
server_key_version: "encryptionkey@0"
I20260811 12:46:19.627286 8252 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:46:19.631057 8278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:19.632303 8252 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:46:19.632488 8252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "5b53dc0348f9496fb6f844943934399f"
format_stamp: "Formatted at 2026-08-11 12:46:19 on dist-test-slave-f0tn"
server_key: "3fec19a340d063fa915032d6322ca1c1"
server_key_iv: "280628aa09e2c6ae077540631b211e5c"
server_key_version: "encryptionkey@0"
I20260811 12:46:19.632675 8252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:19.652083 8252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:19.656515 8252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:19.656854 8252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:19.657990 8252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:46:19.659459 8252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:19.659538 8252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:19.659655 8252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:19.659711 8252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:19.699651 8252 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:38413
I20260811 12:46:19.699731 8472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:38413 every 8 connection(s)
I20260811 12:46:19.700911 8252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:46:19.701339 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 8252
I20260811 12:46:19.701589 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:46:19.702234 7613 external_mini_cluster.cc:1428] Setting key 15c633896afa49d0bb7a18fc18068beb
Aug 11 12:46:19 dist-test-slave-f0tn krb5kdc[7625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452379, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:19.721685 8478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:19.704142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:46605 (local address 127.7.111.126:40735)
0811 12:46:19.704732 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:19.704738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:19.706023 (+ 1285us) server_negotiation.cc:407] Connection header received
0811 12:46:19.708123 (+ 2100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:19.708127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:19.708194 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:19.708286 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:19.709882 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:19.710764 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:19.712362 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:19.712751 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:19.712836 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:19.716131 (+ 3295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:19.716136 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:19.716140 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:19.716189 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:19.718943 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:19.719609 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:19.719612 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:19.719615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:19.719730 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:19.720095 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:19.720098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:19.720100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:19.720418 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:19.720536 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:19.721178 (+ 642us) server_negotiation.cc:299] Negotiation successful
0811 12:46:19.721341 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":320,"threads_started":1}
I20260811 12:46:19.722792 8481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:19.704830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40735 (local address 127.7.111.67:46605)
0811 12:46:19.705864 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:19.705910 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:19.707862 (+ 1952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:19.708480 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:19.708487 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:19.709013 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:19.709725 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:19.709747 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:19.710886 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:19.710889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:19.712251 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:19.712263 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:19.712913 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:19.712918 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:19.713107 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:19.713729 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:19.713752 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:19.715945 (+ 2193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:19.719058 (+ 3113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:19.719065 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:19.719076 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:19.719516 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:19.719801 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:19.719804 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:19.719806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:19.720015 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:19.720519 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:19.720524 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:19.720828 (+ 304us) client_negotiation.cc:769] Sending connection context
0811 12:46:19.721085 (+ 257us) client_negotiation.cc:240] Negotiation successful
0811 12:46:19.721537 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"thread_start_us":528,"threads_started":1}
I20260811 12:46:19.724133 8474 heartbeater.cc:347] Connected to a master server at 127.7.111.126:40735
I20260811 12:46:19.724396 8474 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:19.725112 8474 heartbeater.cc:511] Master 127.7.111.126:40735 requested a full tablet report, sending...
I20260811 12:46:19.726591 7696 ts_manager.cc:194] Registered new tserver with Master: 5b53dc0348f9496fb6f844943934399f (127.7.111.67:38413)
I20260811 12:46:19.727415 7696 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:46605
I20260811 12:46:19.730571 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 12:46:19 dist-test-slave-f0tn krb5kdc[7625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452379, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 12:46:19 dist-test-slave-f0tn krb5kdc[7625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452379, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.111.65@KRBTEST.COM
I20260811 12:46:19.791344 8486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:19.775201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37408 (local address 127.7.111.65:44613)
0811 12:46:19.775978 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:19.775988 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:19.776088 (+ 100us) server_negotiation.cc:407] Connection header received
0811 12:46:19.776300 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:19.776308 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:19.776531 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:19.776706 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:19.778416 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:19.779503 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:19.780872 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:19.781565 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:19.781692 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:19.785651 (+ 3959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:19.785671 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:19.785686 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:19.785726 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:19.788536 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:19.789162 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:19.789171 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:19.789173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:19.789304 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:19.789732 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:19.789736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:19.789738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:19.790126 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:19.790440 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:19.790747 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 12:46:19.791012 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":357,"threads_started":1}
W20260811 12:46:19.792606 7870 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37408
Aug 11 12:46:19 dist-test-slave-f0tn krb5kdc[7625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 12:46:19 dist-test-slave-f0tn krb5kdc[7625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.65@KRBTEST.COM
I20260811 12:46:19.844314 8486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:19.830569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37424 (local address 127.7.111.65:44613)
0811 12:46:19.830841 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:19.830848 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:19.830994 (+ 146us) server_negotiation.cc:407] Connection header received
0811 12:46:19.831147 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:19.831151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:19.831251 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:19.831340 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:19.832639 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:19.833344 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:19.834914 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:19.835309 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:19.835404 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:19.839631 (+ 4227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:19.839636 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:19.839639 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:19.839672 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:19.842187 (+ 2515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:19.842762 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:19.842766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:19.842768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:19.842884 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:19.843137 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:19.843140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:19.843142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:19.843375 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:19.843550 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:19.843814 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 12:46:19.843975 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 12:46:19.845950 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 7793
I20260811 12:46:19.862586 8486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:19.847158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37430 (local address 127.7.111.65:44613)
0811 12:46:19.847353 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:19.847358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:19.847584 (+ 226us) server_negotiation.cc:407] Connection header received
0811 12:46:19.847821 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:19.847825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:19.847881 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:19.847960 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:19.849420 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:19.850372 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:19.852191 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:19.852608 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:19.852726 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:19.856509 (+ 3783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:19.856515 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:19.856518 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:19.856550 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:19.859457 (+ 2907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:19.860270 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:19.860275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:19.860279 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:19.860459 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:19.860869 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:19.860874 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:19.860877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:19.861225 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:19.861490 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:19.861877 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 12:46:19.862101 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:46:19.863463 8008 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:19.948200 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 7793
I20260811 12:46:19.971591 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 8028
Aug 11 12:46:19 dist-test-slave-f0tn krb5kdc[7625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.66@KRBTEST.COM
I20260811 12:46:19.987272 8499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:19.972529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44944 (local address 127.7.111.66:34287)
0811 12:46:19.973105 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:19.973112 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:19.973140 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:46:19.973212 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:19.973219 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:19.973465 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:19.973610 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:19.974780 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:19.975865 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:19.976994 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:19.977764 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:19.977859 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:19.981797 (+ 3938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:19.981809 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:19.981824 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:19.981867 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:19.984529 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:19.985147 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:19.985155 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:19.985157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:19.985293 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:19.985722 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:19.985726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:19.985728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:19.986067 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:19.986377 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:19.986654 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 12:46:19.986923 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":277,"threads_started":1}
I20260811 12:46:19.988508 8232 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:20.071087 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 8028
I20260811 12:46:20.096258 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 8252
Aug 11 12:46:20 dist-test-slave-f0tn krb5kdc[7625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.67@KRBTEST.COM
I20260811 12:46:20.112064 8502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:20.097339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52780 (local address 127.7.111.67:38413)
0811 12:46:20.098099 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:20.098106 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:20.098139 (+ 33us) server_negotiation.cc:407] Connection header received
0811 12:46:20.098207 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:20.098214 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:20.098428 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:20.098566 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:20.099808 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:20.100861 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:20.102137 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:20.102709 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:20.102806 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:20.106754 (+ 3948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:20.106765 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:20.106780 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:20.106818 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:20.109707 (+ 2889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:20.110319 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:20.110326 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:20.110328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:20.110450 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:20.110800 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:20.110803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:20.110804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:20.111116 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:20.111361 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:20.111573 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 12:46:20.111777 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":350,"threads_started":1}
I20260811 12:46:20.113341 8467 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:20.195036 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 8252
I20260811 12:46:20.222657 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 7643
I20260811 12:46:20.223973 7741 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:20.328608 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 7643
2026-08-11T12:46:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4431 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:46:20 dist-test-slave-f0tn krb5kdc[8509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:46:20 dist-test-slave-f0tn krb5kdc[8509](info): set up 2 sockets
Aug 11 12:46:20 dist-test-slave-f0tn krb5kdc[8509](info): commencing operation
krb5kdc: starting...
W20260811 12:46:21.549243 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.005s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:46:21 dist-test-slave-f0tn krb5kdc[8509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:46:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:46:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:46:21.750070 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:44237
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:34477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:44237
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:22.083318 8531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:22.083652 8531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:22.083802 8531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:22.091311 8531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:46:22.091421 8531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:22.091495 8531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:22.091569 8531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:46:22.091634 8531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:46:22.101615 8531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:34477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:44237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:44237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:22.102977 8531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:22.104307 8531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:22.115898 8540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:22.115857 8541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:22.115947 8544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:22.116885 8531 server_base.cc:1034] running on GCE node
I20260811 12:46:22.117681 8531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:22.118988 8531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:22.120190 8531 hybrid_clock.cc:648] HybridClock initialized: now 1786452382120140 us; error 40 us; skew 500 ppm
I20260811 12:46:22.120512 8531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:22 dist-test-slave-f0tn krb5kdc[8509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452382, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:22.123450 8531 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:46:22.124902 8531 webserver.cc:460] Webserver started at http://127.7.111.126:34677/ using document root <none> and password file <none>
I20260811 12:46:22.125608 8531 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:22.125723 8531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:22.126022 8531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:22.128859 8531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "c88d34d247e446dfb60a09698125b1fd"
format_stamp: "Formatted at 2026-08-11 12:46:22 on dist-test-slave-f0tn"
server_key: "491770707348c7076b36fa2dd144f497"
server_key_iv: "833c8a5fa85e91181576b90603c47cf4"
server_key_version: "encryptionkey@0"
I20260811 12:46:22.129637 8531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "c88d34d247e446dfb60a09698125b1fd"
format_stamp: "Formatted at 2026-08-11 12:46:22 on dist-test-slave-f0tn"
server_key: "491770707348c7076b36fa2dd144f497"
server_key_iv: "833c8a5fa85e91181576b90603c47cf4"
server_key_version: "encryptionkey@0"
I20260811 12:46:22.134531 8531 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 12:46:22.137861 8554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:22.139119 8531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 12:46:22.139276 8531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "c88d34d247e446dfb60a09698125b1fd"
format_stamp: "Formatted at 2026-08-11 12:46:22 on dist-test-slave-f0tn"
server_key: "491770707348c7076b36fa2dd144f497"
server_key_iv: "833c8a5fa85e91181576b90603c47cf4"
server_key_version: "encryptionkey@0"
I20260811 12:46:22.139449 8531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:22.155853 8531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:22.159585 8531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:22.159850 8531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:22.176879 8531 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:44237
I20260811 12:46:22.176899 8642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:44237 every 8 connection(s)
I20260811 12:46:22.178215 8531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:46:22.181761 8644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:22.186610 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 8531
I20260811 12:46:22.186852 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:46:22.187640 7613 external_mini_cluster.cc:1428] Setting key 633d5a5a5962ed2d411cd007fb6edebd
I20260811 12:46:22.192823 8644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c88d34d247e446dfb60a09698125b1fd: Bootstrap starting.
I20260811 12:46:22.195796 8644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c88d34d247e446dfb60a09698125b1fd: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:22.196734 8644 log.cc:824] T 00000000000000000000000000000000 P c88d34d247e446dfb60a09698125b1fd: Log is configured to *not* fsync() on all Append() calls
I20260811 12:46:22.199452 8644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c88d34d247e446dfb60a09698125b1fd: No bootstrap required, opened a new log
Aug 11 12:46:22 dist-test-slave-f0tn krb5kdc[8509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:22.207428 8644 raft_consensus.cc:348] T 00000000000000000000000000000000 P c88d34d247e446dfb60a09698125b1fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c88d34d247e446dfb60a09698125b1fd" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 44237 } }
I20260811 12:46:22.207847 8644 raft_consensus.cc:374] T 00000000000000000000000000000000 P c88d34d247e446dfb60a09698125b1fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:22.207962 8644 raft_consensus.cc:729] T 00000000000000000000000000000000 P c88d34d247e446dfb60a09698125b1fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c88d34d247e446dfb60a09698125b1fd, State: Initialized, Role: FOLLOWER
I20260811 12:46:22.208544 8644 consensus_queue.cc:260] T 00000000000000000000000000000000 P c88d34d247e446dfb60a09698125b1fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c88d34d247e446dfb60a09698125b1fd" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 44237 } }
I20260811 12:46:22.208796 8644 raft_consensus.cc:388] T 00000000000000000000000000000000 P c88d34d247e446dfb60a09698125b1fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:46:22.208945 8644 raft_consensus.cc:482] T 00000000000000000000000000000000 P c88d34d247e446dfb60a09698125b1fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:46:22.208977 8649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:22.189725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38166 (local address 127.7.111.126:44237)
0811 12:46:22.190517 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:22.190529 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:22.190567 (+ 38us) server_negotiation.cc:407] Connection header received
0811 12:46:22.192226 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:22.192249 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:22.192604 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:22.192934 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:22.194379 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:22.195634 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:22.196848 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:22.197604 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:22.197783 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:22.201522 (+ 3739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:22.201529 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:22.201543 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:22.201585 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:22.204980 (+ 3395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:22.205654 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:22.205660 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:22.205670 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:22.205854 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:22.206217 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:22.206221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:22.206223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:22.206741 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:22.207023 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:22.207524 (+ 501us) server_negotiation.cc:299] Negotiation successful
0811 12:46:22.207928 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":356,"threads_started":1}
I20260811 12:46:22.209125 8644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c88d34d247e446dfb60a09698125b1fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:22.211826 8644 raft_consensus.cc:504] T 00000000000000000000000000000000 P c88d34d247e446dfb60a09698125b1fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c88d34d247e446dfb60a09698125b1fd" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 44237 } }
I20260811 12:46:22.212306 8644 leader_election.cc:302] T 00000000000000000000000000000000 P c88d34d247e446dfb60a09698125b1fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c88d34d247e446dfb60a09698125b1fd; no voters:
I20260811 12:46:22.212796 8644 leader_election.cc:288] T 00000000000000000000000000000000 P c88d34d247e446dfb60a09698125b1fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:46:22.212920 8653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c88d34d247e446dfb60a09698125b1fd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:46:22.213665 8653 raft_consensus.cc:686] T 00000000000000000000000000000000 P c88d34d247e446dfb60a09698125b1fd [term 1 LEADER]: Becoming Leader. State: Replica: c88d34d247e446dfb60a09698125b1fd, State: Running, Role: LEADER
I20260811 12:46:22.214198 8653 consensus_queue.cc:237] T 00000000000000000000000000000000 P c88d34d247e446dfb60a09698125b1fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c88d34d247e446dfb60a09698125b1fd" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 44237 } }
I20260811 12:46:22.214874 8644 sys_catalog.cc:564] T 00000000000000000000000000000000 P c88d34d247e446dfb60a09698125b1fd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:46:22.216943 8655 sys_catalog.cc:455] T 00000000000000000000000000000000 P c88d34d247e446dfb60a09698125b1fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader c88d34d247e446dfb60a09698125b1fd. Latest consensus state: current_term: 1 leader_uuid: "c88d34d247e446dfb60a09698125b1fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c88d34d247e446dfb60a09698125b1fd" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 44237 } } }
I20260811 12:46:22.217345 8655 sys_catalog.cc:458] T 00000000000000000000000000000000 P c88d34d247e446dfb60a09698125b1fd [sys.catalog]: This master's current role is: LEADER
I20260811 12:46:22.217264 8654 sys_catalog.cc:455] T 00000000000000000000000000000000 P c88d34d247e446dfb60a09698125b1fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c88d34d247e446dfb60a09698125b1fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c88d34d247e446dfb60a09698125b1fd" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 44237 } } }
I20260811 12:46:22.217614 8654 sys_catalog.cc:458] T 00000000000000000000000000000000 P c88d34d247e446dfb60a09698125b1fd [sys.catalog]: This master's current role is: LEADER
I20260811 12:46:22.218580 8664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:46:22.224042 8664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:46:22.233013 8664 catalog_manager.cc:1347] Generated new cluster ID: 3429b3d2bf0344ffad4171fca81e7d27
I20260811 12:46:22.233132 8664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:46:22.251709 8664 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:46:22.253577 8664 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:46:22.260665 8664 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c88d34d247e446dfb60a09698125b1fd: Generated new TSK 0
I20260811 12:46:22.261832 8664 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.111.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
I20260811 12:46:22.307221 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:44237
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:34477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:22.641018 8691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:22.641290 8691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:22.641461 8691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:22.648689 8691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:22.648792 8691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:22.648943 8691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:46:22.658564 8691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:34477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:44237
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:22.659861 8691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:22.661173 8691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:22.674271 8699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:22.674201 8706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:22.674247 8703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:22.675390 8691 server_base.cc:1034] running on GCE node
I20260811 12:46:22.675973 8691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:22.676865 8691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:22.678061 8691 hybrid_clock.cc:648] HybridClock initialized: now 1786452382678016 us; error 36 us; skew 500 ppm
I20260811 12:46:22.678367 8691 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:22 dist-test-slave-f0tn krb5kdc[8509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452382, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:22.681301 8691 init.cc:377] Logged in from keytab as kudu/127.7.111.65@KRBTEST.COM (short username kudu)
I20260811 12:46:22.682698 8691 webserver.cc:460] Webserver started at http://127.7.111.65:37393/ using document root <none> and password file <none>
I20260811 12:46:22.683363 8691 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:22.683477 8691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:22.683768 8691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:22.686607 8691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "7db60368d4074410875e8a7b7dbc958f"
format_stamp: "Formatted at 2026-08-11 12:46:22 on dist-test-slave-f0tn"
server_key: "6da03f70785281f0db98344e4d8a4e9b"
server_key_iv: "e8ff0c61e9ed9601db5f39ad8874f4fb"
server_key_version: "encryptionkey@0"
I20260811 12:46:22.687364 8691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "7db60368d4074410875e8a7b7dbc958f"
format_stamp: "Formatted at 2026-08-11 12:46:22 on dist-test-slave-f0tn"
server_key: "6da03f70785281f0db98344e4d8a4e9b"
server_key_iv: "e8ff0c61e9ed9601db5f39ad8874f4fb"
server_key_version: "encryptionkey@0"
I20260811 12:46:22.692369 8691 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:46:22.695655 8717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:22.696869 8691 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:46:22.697023 8691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "7db60368d4074410875e8a7b7dbc958f"
format_stamp: "Formatted at 2026-08-11 12:46:22 on dist-test-slave-f0tn"
server_key: "6da03f70785281f0db98344e4d8a4e9b"
server_key_iv: "e8ff0c61e9ed9601db5f39ad8874f4fb"
server_key_version: "encryptionkey@0"
I20260811 12:46:22.697203 8691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:22.737179 8691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:22.741065 8691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:22.741355 8691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:22.742416 8691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:46:22.743853 8691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:22.743933 8691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:22.744061 8691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:22.744127 8691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:22.775044 8691 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:46721
I20260811 12:46:22.775097 8923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:46721 every 8 connection(s)
I20260811 12:46:22.776408 8691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:46:22.786784 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 8691
I20260811 12:46:22.787003 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:46:22.787808 7613 external_mini_cluster.cc:1428] Setting key 478a155a5278abdaf1b21e6467a064b1
Aug 11 12:46:22 dist-test-slave-f0tn krb5kdc[8509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452382, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:22.796830 8926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:22.779022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:47365 (local address 127.7.111.126:44237)
0811 12:46:22.779464 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:22.779468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:22.780423 (+ 955us) server_negotiation.cc:407] Connection header received
0811 12:46:22.782346 (+ 1923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:22.782352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:22.782444 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:22.782546 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:22.784099 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:22.784828 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:22.786403 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:22.786809 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:22.786940 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:22.791203 (+ 4263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:22.791208 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:22.791211 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:22.791245 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:22.793765 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:22.794592 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:22.794596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:22.794598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:22.794713 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:22.795134 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:22.795137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:22.795139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:22.795398 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:22.795508 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:22.796373 (+ 865us) server_negotiation.cc:299] Negotiation successful
0811 12:46:22.796555 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":226,"threads_started":1}
I20260811 12:46:22.798099 8927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:22.779575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:44237 (local address 127.7.111.65:47365)
0811 12:46:22.780261 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:22.780299 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:22.782073 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:22.782822 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:22.782830 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:22.783342 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:22.783925 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:22.783943 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:22.784983 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:22.784987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:22.786261 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:22.786274 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:22.787070 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:22.787079 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:22.787403 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:22.788132 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:22.788156 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:22.790971 (+ 2815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:22.793903 (+ 2932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:22.793912 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:22.793929 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:22.794488 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:22.794806 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:22.794810 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:22.794813 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:22.795046 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:22.795517 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:22.795524 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:22.795944 (+ 420us) client_negotiation.cc:769] Sending connection context
0811 12:46:22.796258 (+ 314us) client_negotiation.cc:240] Negotiation successful
0811 12:46:22.796791 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":260,"threads_started":1}
I20260811 12:46:22.799676 8924 heartbeater.cc:347] Connected to a master server at 127.7.111.126:44237
I20260811 12:46:22.799978 8924 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:22.800835 8924 heartbeater.cc:511] Master 127.7.111.126:44237 requested a full tablet report, sending...
I20260811 12:46:22.803036 8591 ts_manager.cc:194] Registered new tserver with Master: 7db60368d4074410875e8a7b7dbc958f (127.7.111.65:46721)
I20260811 12:46:22.804556 8591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:47365
WARNING: no policy specified for kudu/127.7.111.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
I20260811 12:46:22.831696 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:44237
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:34477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:23.155762 8938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:23.156090 8938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:23.156239 8938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:23.163568 8938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:23.163703 8938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:23.163884 8938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:46:23.173930 8938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:34477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:44237
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:23.175202 8938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:23.176545 8938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:23.188970 8953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:23.189041 8951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:23.190049 8956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:23.191646 8938 server_base.cc:1034] running on GCE node
I20260811 12:46:23.192158 8938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:23.192986 8938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:23.194165 8938 hybrid_clock.cc:648] HybridClock initialized: now 1786452383194113 us; error 50 us; skew 500 ppm
I20260811 12:46:23.194468 8938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:23 dist-test-slave-f0tn krb5kdc[8509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452383, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:23.203207 8938 init.cc:377] Logged in from keytab as kudu/127.7.111.66@KRBTEST.COM (short username kudu)
I20260811 12:46:23.204617 8938 webserver.cc:460] Webserver started at http://127.7.111.66:41747/ using document root <none> and password file <none>
I20260811 12:46:23.205320 8938 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:23.205474 8938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:23.205839 8938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:23.208762 8938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "af793fda75cf4ee3afc59e2db5b41aeb"
format_stamp: "Formatted at 2026-08-11 12:46:23 on dist-test-slave-f0tn"
server_key: "534622e9854bb3c69e5456b077728263"
server_key_iv: "3a3b03ccc5dc25513ffd3f4fdf0476e5"
server_key_version: "encryptionkey@0"
I20260811 12:46:23.209560 8938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "af793fda75cf4ee3afc59e2db5b41aeb"
format_stamp: "Formatted at 2026-08-11 12:46:23 on dist-test-slave-f0tn"
server_key: "534622e9854bb3c69e5456b077728263"
server_key_iv: "3a3b03ccc5dc25513ffd3f4fdf0476e5"
server_key_version: "encryptionkey@0"
I20260811 12:46:23.214548 8938 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 12:46:23.218147 8965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:23.219451 8938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:46:23.219599 8938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "af793fda75cf4ee3afc59e2db5b41aeb"
format_stamp: "Formatted at 2026-08-11 12:46:23 on dist-test-slave-f0tn"
server_key: "534622e9854bb3c69e5456b077728263"
server_key_iv: "3a3b03ccc5dc25513ffd3f4fdf0476e5"
server_key_version: "encryptionkey@0"
I20260811 12:46:23.219774 8938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:23.262509 8938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:23.266834 8938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:23.267108 8938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:23.268108 8938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:46:23.269577 8938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:23.269651 8938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:23.269742 8938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:23.269785 8938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:23.300953 8938 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:38947
I20260811 12:46:23.301023 9158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:38947 every 8 connection(s)
I20260811 12:46:23.302392 8938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:46:23.309237 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 8938
I20260811 12:46:23.309458 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:46:23.309891 7613 external_mini_cluster.cc:1428] Setting key 796c08c3af6199ecb47e7c9a5d58a849
Aug 11 12:46:23 dist-test-slave-f0tn krb5kdc[8509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452383, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:23.323017 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:23.305039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:46873 (local address 127.7.111.126:44237)
0811 12:46:23.305532 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:23.305537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:23.306434 (+ 897us) server_negotiation.cc:407] Connection header received
0811 12:46:23.308287 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:23.308293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:23.308388 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:23.308487 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:23.310112 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.310825 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.312564 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.313115 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.313241 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:23.316801 (+ 3560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:23.316806 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:23.316810 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:23.316850 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:23.319754 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:23.320638 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:23.320642 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:23.320645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:23.320784 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:23.321236 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:23.321238 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:23.321240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:23.321541 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:23.321652 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:23.322497 (+ 845us) server_negotiation.cc:299] Negotiation successful
0811 12:46:23.322680 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":260,"threads_started":1}
I20260811 12:46:23.324542 9164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:23.305550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:44237 (local address 127.7.111.66:46873)
0811 12:46:23.306267 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:23.306307 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:23.308007 (+ 1700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:23.308738 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:23.308746 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:23.309271 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:23.309929 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:23.309953 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.310991 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.310996 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:23.312396 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:23.312412 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.313344 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.313349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:23.313605 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:23.314259 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:23.314282 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:23.316573 (+ 2291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:23.319893 (+ 3320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:23.319903 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:23.319922 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:23.320533 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:23.320886 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:23.320891 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:23.320894 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:23.321141 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:23.321666 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:23.321681 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:23.322121 (+ 440us) client_negotiation.cc:769] Sending connection context
0811 12:46:23.322427 (+ 306us) client_negotiation.cc:240] Negotiation successful
0811 12:46:23.323014 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":274,"threads_started":1}
I20260811 12:46:23.326031 9159 heartbeater.cc:347] Connected to a master server at 127.7.111.126:44237
I20260811 12:46:23.326375 9159 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:23.327332 9159 heartbeater.cc:511] Master 127.7.111.126:44237 requested a full tablet report, sending...
I20260811 12:46:23.328958 8591 ts_manager.cc:194] Registered new tserver with Master: af793fda75cf4ee3afc59e2db5b41aeb (127.7.111.66:38947)
I20260811 12:46:23.329828 8591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:46873
WARNING: no policy specified for kudu/127.7.111.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
I20260811 12:46:23.358448 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:44237
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:34477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:23.684859 9168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:23.685165 9168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:23.685313 9168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:23.692641 9168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:23.692754 9168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:23.692883 9168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:46:23.702883 9168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:34477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:44237
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:23.704284 9168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:23.705785 9168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:23.719457 9181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:23.719497 9184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:23.719465 9177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:23.720219 9168 server_base.cc:1034] running on GCE node
I20260811 12:46:23.720860 9168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:23.721920 9168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:23.723104 9168 hybrid_clock.cc:648] HybridClock initialized: now 1786452383723051 us; error 58 us; skew 500 ppm
I20260811 12:46:23.723441 9168 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:23 dist-test-slave-f0tn krb5kdc[8509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452383, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:23.726938 9168 init.cc:377] Logged in from keytab as kudu/127.7.111.67@KRBTEST.COM (short username kudu)
I20260811 12:46:23.728412 9168 webserver.cc:460] Webserver started at http://127.7.111.67:38875/ using document root <none> and password file <none>
I20260811 12:46:23.729130 9168 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:23.729245 9168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:23.729609 9168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:23.732693 9168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "48a908c7abe54c04b2de55cd36b1d8fc"
format_stamp: "Formatted at 2026-08-11 12:46:23 on dist-test-slave-f0tn"
server_key: "510311dc7e71024c2d81afe24193e63f"
server_key_iv: "43c2fec050f9b0cde017b396f7630c79"
server_key_version: "encryptionkey@0"
I20260811 12:46:23.733541 9168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "48a908c7abe54c04b2de55cd36b1d8fc"
format_stamp: "Formatted at 2026-08-11 12:46:23 on dist-test-slave-f0tn"
server_key: "510311dc7e71024c2d81afe24193e63f"
server_key_iv: "43c2fec050f9b0cde017b396f7630c79"
server_key_version: "encryptionkey@0"
I20260811 12:46:23.738742 9168 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:46:23.742245 9192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:23.743588 9168 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:46:23.743764 9168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "48a908c7abe54c04b2de55cd36b1d8fc"
format_stamp: "Formatted at 2026-08-11 12:46:23 on dist-test-slave-f0tn"
server_key: "510311dc7e71024c2d81afe24193e63f"
server_key_iv: "43c2fec050f9b0cde017b396f7630c79"
server_key_version: "encryptionkey@0"
I20260811 12:46:23.743952 9168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:23.764868 9168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:23.768882 9168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:23.769165 9168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:23.770220 9168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:46:23.771575 9168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:23.771658 9168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:23.771792 9168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:23.771847 9168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:23.804744 9168 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:46157
I20260811 12:46:23.804781 9371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:46157 every 8 connection(s)
I20260811 12:46:23.806190 9168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:46:23.807407 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 9168
I20260811 12:46:23.807657 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:46:23.807848 8924 heartbeater.cc:503] Master 127.7.111.126:44237 was elected leader, sending a full tablet report...
I20260811 12:46:23.808193 7613 external_mini_cluster.cc:1428] Setting key 7b293bf6545b286607ab85c86bb9cc15
Aug 11 12:46:23 dist-test-slave-f0tn krb5kdc[8509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452383, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:23.827597 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:23.809055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:47469 (local address 127.7.111.126:44237)
0811 12:46:23.809270 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:23.809276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:23.810886 (+ 1610us) server_negotiation.cc:407] Connection header received
0811 12:46:23.812949 (+ 2063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:23.812953 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:23.813028 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:23.813110 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:23.814798 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.815643 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.817325 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.817755 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.817848 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:23.821263 (+ 3415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:23.821268 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:23.821271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:23.821304 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:23.824562 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:23.825285 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:23.825291 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:23.825294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:23.825448 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:23.825864 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:23.825870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:23.825873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:23.826154 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:23.826301 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:23.827008 (+ 707us) server_negotiation.cc:299] Negotiation successful
0811 12:46:23.827274 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:46:23.828565 9379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:23.809802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:44237 (local address 127.7.111.67:47469)
0811 12:46:23.810722 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:23.810781 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:23.812703 (+ 1922us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:23.813295 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:23.813303 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:23.813870 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:23.814650 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:23.814679 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.815786 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.815790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:23.817215 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:23.817229 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.817971 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.817976 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:23.818180 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:23.818806 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:23.818829 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:23.821069 (+ 2240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:23.824690 (+ 3621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:23.824697 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:23.824710 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:23.825161 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:23.825551 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:23.825556 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:23.825560 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:23.825755 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:23.826303 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:23.826311 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:23.826643 (+ 332us) client_negotiation.cc:769] Sending connection context
0811 12:46:23.826891 (+ 248us) client_negotiation.cc:240] Negotiation successful
0811 12:46:23.827438 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":374,"threads_started":1}
I20260811 12:46:23.829831 9373 heartbeater.cc:347] Connected to a master server at 127.7.111.126:44237
I20260811 12:46:23.830180 9373 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:23.830899 9373 heartbeater.cc:511] Master 127.7.111.126:44237 requested a full tablet report, sending...
I20260811 12:46:23.832387 8584 ts_manager.cc:194] Registered new tserver with Master: 48a908c7abe54c04b2de55cd36b1d8fc (127.7.111.67:46157)
I20260811 12:46:23.833252 8584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:47469
I20260811 12:46:23.836261 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:46:23.855959 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:23.843408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38170 (local address 127.7.111.126:44237)
0811 12:46:23.843568 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:23.843573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:23.844069 (+ 496us) server_negotiation.cc:407] Connection header received
0811 12:46:23.844217 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:23.844221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:23.844299 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:23.844379 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:23.846085 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.846803 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.848119 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.848569 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.848661 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:23.851030 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:23.851034 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:23.851038 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:23.851067 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:23.853700 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:23.854307 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:23.854310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:23.854313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:23.854429 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:23.854701 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:23.854704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:23.854705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:23.855085 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:23.855286 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:23.855544 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 12:46:23.855714 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:46:23.863787 8584 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:46:23.871045 8584 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:46:23.911038 9403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:23.885944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:46721 (local address 127.0.0.1:51886)
0811 12:46:23.900375 (+ 14431us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:23.900395 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:23.900525 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:23.902962 (+ 2437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:23.902970 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:23.903001 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:23.903691 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:23.903712 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.906270 (+ 2558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.906275 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:23.908599 (+ 2324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:23.908613 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.910061 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.910068 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:23.910261 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:23.910317 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 12:46:23.910441 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 12:46:23.910640 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14302,"thread_start_us":11460,"threads_started":1}
I20260811 12:46:23.911383 9405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:23.885578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59676 (local address 127.7.111.66:38947)
0811 12:46:23.902716 (+ 17138us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:23.902725 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:23.902753 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:46:23.902851 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:23.902858 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:23.903081 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:23.903281 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:23.904137 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.905489 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.908370 (+ 2881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.909924 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.910060 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:23.910250 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:23.910829 (+ 579us) server_negotiation.cc:299] Negotiation successful
0811 12:46:23.911000 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16964,"thread_start_us":16815,"threads_started":1}
I20260811 12:46:23.911978 9409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:23.898710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41834 (local address 127.7.111.67:46157)
0811 12:46:23.902211 (+ 3501us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:23.902220 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:23.903249 (+ 1029us) server_negotiation.cc:407] Connection header received
0811 12:46:23.903478 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:23.903486 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:23.903704 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:23.903890 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:23.904976 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.906511 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.908951 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.910330 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.910457 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:23.910672 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:23.911180 (+ 508us) server_negotiation.cc:299] Negotiation successful
0811 12:46:23.911341 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3312,"thread_start_us":3090,"threads_started":1}
I20260811 12:46:23.912093 9407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:23.886160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51886 (local address 127.7.111.65:46721)
0811 12:46:23.901394 (+ 15234us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:23.901901 (+ 507us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:23.901935 (+ 34us) server_negotiation.cc:407] Connection header received
0811 12:46:23.902070 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:23.902079 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:23.902345 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:23.902547 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:23.904345 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.906116 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.908780 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.909824 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.909989 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:23.910278 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:23.911591 (+ 1313us) server_negotiation.cc:299] Negotiation successful
0811 12:46:23.911764 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15009,"thread_start_us":15490,"threads_started":1}
I20260811 12:46:23.911348 9406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:23.885475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:38947 (local address 127.0.0.1:59676)
0811 12:46:23.897175 (+ 11700us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:23.897239 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:23.898604 (+ 1365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:23.903351 (+ 4747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:23.903356 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:23.903387 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:23.903957 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:23.903972 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.905607 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.905612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:23.908183 (+ 2571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:23.908198 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.910437 (+ 2239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.910445 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:23.910633 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:23.910666 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:46:23.910764 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 12:46:23.910890 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11368,"thread_start_us":15108,"threads_started":1}
I20260811 12:46:23.911669 9408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:23.898328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:46157 (local address 127.0.0.1:41834)
0811 12:46:23.903103 (+ 4775us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:23.903123 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:23.903265 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:23.904073 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:23.904077 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:23.904102 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:23.904787 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:23.904803 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.906645 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.906649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:23.908790 (+ 2141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:23.908805 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.910593 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.910601 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:23.910818 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:23.910852 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 12:46:23.910939 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 12:46:23.911058 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4631,"thread_start_us":4422,"threads_started":1}
I20260811 12:46:23.916162 8796 tablet_service.cc:1467] Processing CreateTablet for tablet a03397e30ca347468ab7add66e04d4c6 (DEFAULT_TABLE table=test-table [id=df3197fa784c4dcc9e76bb2c944f59b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:23.916438 9270 tablet_service.cc:1467] Processing CreateTablet for tablet a03397e30ca347468ab7add66e04d4c6 (DEFAULT_TABLE table=test-table [id=df3197fa784c4dcc9e76bb2c944f59b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:23.916640 9043 tablet_service.cc:1467] Processing CreateTablet for tablet a03397e30ca347468ab7add66e04d4c6 (DEFAULT_TABLE table=test-table [id=df3197fa784c4dcc9e76bb2c944f59b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:23.917714 8796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a03397e30ca347468ab7add66e04d4c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:23.917944 9270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a03397e30ca347468ab7add66e04d4c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:23.918239 9043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a03397e30ca347468ab7add66e04d4c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:23.929625 9410 tablet_bootstrap.cc:492] T a03397e30ca347468ab7add66e04d4c6 P 7db60368d4074410875e8a7b7dbc958f: Bootstrap starting.
I20260811 12:46:23.932572 9411 tablet_bootstrap.cc:492] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc: Bootstrap starting.
I20260811 12:46:23.934144 9412 tablet_bootstrap.cc:492] T a03397e30ca347468ab7add66e04d4c6 P af793fda75cf4ee3afc59e2db5b41aeb: Bootstrap starting.
I20260811 12:46:23.934203 9410 tablet_bootstrap.cc:654] T a03397e30ca347468ab7add66e04d4c6 P 7db60368d4074410875e8a7b7dbc958f: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:23.935647 9410 log.cc:824] T a03397e30ca347468ab7add66e04d4c6 P 7db60368d4074410875e8a7b7dbc958f: Log is configured to *not* fsync() on all Append() calls
I20260811 12:46:23.936465 9411 tablet_bootstrap.cc:654] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:23.937340 9412 tablet_bootstrap.cc:654] T a03397e30ca347468ab7add66e04d4c6 P af793fda75cf4ee3afc59e2db5b41aeb: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:23.937765 9411 log.cc:824] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc: Log is configured to *not* fsync() on all Append() calls
I20260811 12:46:23.938556 9412 log.cc:824] T a03397e30ca347468ab7add66e04d4c6 P af793fda75cf4ee3afc59e2db5b41aeb: Log is configured to *not* fsync() on all Append() calls
I20260811 12:46:23.939085 9410 tablet_bootstrap.cc:492] T a03397e30ca347468ab7add66e04d4c6 P 7db60368d4074410875e8a7b7dbc958f: No bootstrap required, opened a new log
I20260811 12:46:23.939428 9410 ts_tablet_manager.cc:1397] T a03397e30ca347468ab7add66e04d4c6 P 7db60368d4074410875e8a7b7dbc958f: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260811 12:46:23.941418 9411 tablet_bootstrap.cc:492] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc: No bootstrap required, opened a new log
I20260811 12:46:23.941689 9411 ts_tablet_manager.cc:1397] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260811 12:46:23.947476 9412 tablet_bootstrap.cc:492] T a03397e30ca347468ab7add66e04d4c6 P af793fda75cf4ee3afc59e2db5b41aeb: No bootstrap required, opened a new log
I20260811 12:46:23.947786 9412 ts_tablet_manager.cc:1397] T a03397e30ca347468ab7add66e04d4c6 P af793fda75cf4ee3afc59e2db5b41aeb: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.006s
I20260811 12:46:23.951097 9410 raft_consensus.cc:348] T a03397e30ca347468ab7add66e04d4c6 P 7db60368d4074410875e8a7b7dbc958f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af793fda75cf4ee3afc59e2db5b41aeb" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 38947 } } peers { permanent_uuid: "7db60368d4074410875e8a7b7dbc958f" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 46721 } } peers { permanent_uuid: "48a908c7abe54c04b2de55cd36b1d8fc" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 46157 } }
I20260811 12:46:23.951532 9410 raft_consensus.cc:374] T a03397e30ca347468ab7add66e04d4c6 P 7db60368d4074410875e8a7b7dbc958f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:23.951650 9410 raft_consensus.cc:729] T a03397e30ca347468ab7add66e04d4c6 P 7db60368d4074410875e8a7b7dbc958f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7db60368d4074410875e8a7b7dbc958f, State: Initialized, Role: FOLLOWER
I20260811 12:46:23.952359 9410 consensus_queue.cc:260] T a03397e30ca347468ab7add66e04d4c6 P 7db60368d4074410875e8a7b7dbc958f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af793fda75cf4ee3afc59e2db5b41aeb" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 38947 } } peers { permanent_uuid: "7db60368d4074410875e8a7b7dbc958f" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 46721 } } peers { permanent_uuid: "48a908c7abe54c04b2de55cd36b1d8fc" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 46157 } }
I20260811 12:46:23.952440 9411 raft_consensus.cc:348] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af793fda75cf4ee3afc59e2db5b41aeb" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 38947 } } peers { permanent_uuid: "7db60368d4074410875e8a7b7dbc958f" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 46721 } } peers { permanent_uuid: "48a908c7abe54c04b2de55cd36b1d8fc" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 46157 } }
I20260811 12:46:23.952898 9411 raft_consensus.cc:374] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:23.953088 9411 raft_consensus.cc:729] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48a908c7abe54c04b2de55cd36b1d8fc, State: Initialized, Role: FOLLOWER
I20260811 12:46:23.953862 9411 consensus_queue.cc:260] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af793fda75cf4ee3afc59e2db5b41aeb" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 38947 } } peers { permanent_uuid: "7db60368d4074410875e8a7b7dbc958f" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 46721 } } peers { permanent_uuid: "48a908c7abe54c04b2de55cd36b1d8fc" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 46157 } }
I20260811 12:46:23.954378 9410 ts_tablet_manager.cc:1428] T a03397e30ca347468ab7add66e04d4c6 P 7db60368d4074410875e8a7b7dbc958f: Time spent starting tablet: real 0.014s user 0.005s sys 0.008s
I20260811 12:46:23.955145 9373 heartbeater.cc:503] Master 127.7.111.126:44237 was elected leader, sending a full tablet report...
I20260811 12:46:23.955695 9411 ts_tablet_manager.cc:1428] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc: Time spent starting tablet: real 0.014s user 0.008s sys 0.005s
I20260811 12:46:23.958266 9412 raft_consensus.cc:348] T a03397e30ca347468ab7add66e04d4c6 P af793fda75cf4ee3afc59e2db5b41aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af793fda75cf4ee3afc59e2db5b41aeb" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 38947 } } peers { permanent_uuid: "7db60368d4074410875e8a7b7dbc958f" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 46721 } } peers { permanent_uuid: "48a908c7abe54c04b2de55cd36b1d8fc" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 46157 } }
I20260811 12:46:23.958639 9412 raft_consensus.cc:374] T a03397e30ca347468ab7add66e04d4c6 P af793fda75cf4ee3afc59e2db5b41aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:23.958750 9412 raft_consensus.cc:729] T a03397e30ca347468ab7add66e04d4c6 P af793fda75cf4ee3afc59e2db5b41aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af793fda75cf4ee3afc59e2db5b41aeb, State: Initialized, Role: FOLLOWER
I20260811 12:46:23.959296 9412 consensus_queue.cc:260] T a03397e30ca347468ab7add66e04d4c6 P af793fda75cf4ee3afc59e2db5b41aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af793fda75cf4ee3afc59e2db5b41aeb" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 38947 } } peers { permanent_uuid: "7db60368d4074410875e8a7b7dbc958f" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 46721 } } peers { permanent_uuid: "48a908c7abe54c04b2de55cd36b1d8fc" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 46157 } }
I20260811 12:46:23.959959 9424 raft_consensus.cc:482] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:46:23.960353 9159 heartbeater.cc:503] Master 127.7.111.126:44237 was elected leader, sending a full tablet report...
I20260811 12:46:23.960950 9412 ts_tablet_manager.cc:1428] T a03397e30ca347468ab7add66e04d4c6 P af793fda75cf4ee3afc59e2db5b41aeb: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260811 12:46:23.961405 9424 raft_consensus.cc:504] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af793fda75cf4ee3afc59e2db5b41aeb" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 38947 } } peers { permanent_uuid: "7db60368d4074410875e8a7b7dbc958f" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 46721 } } peers { permanent_uuid: "48a908c7abe54c04b2de55cd36b1d8fc" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 46157 } }
I20260811 12:46:23.963186 9424 leader_election.cc:288] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af793fda75cf4ee3afc59e2db5b41aeb (127.7.111.66:38947), 7db60368d4074410875e8a7b7dbc958f (127.7.111.65:46721)
I20260811 12:46:23.969545 9407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:23.963633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:59573 (local address 127.7.111.65:46721)
0811 12:46:23.963817 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:23.963822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:23.964198 (+ 376us) server_negotiation.cc:407] Connection header received
0811 12:46:23.964406 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:23.964410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:23.964472 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:23.964554 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:23.965067 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.965989 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.967809 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.968638 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.968784 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:23.968859 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:23.969172 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 12:46:23.969295 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:46:23.969568 9379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:23.963518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:46721 (local address 127.7.111.67:59573)
0811 12:46:23.964088 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:23.964117 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:23.964237 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:23.964587 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:23.964591 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:23.964620 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:23.964920 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:23.964934 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.966126 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.966132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:23.967642 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:23.967656 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.968785 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.968791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:23.969021 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:23.969047 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 12:46:23.969128 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 12:46:23.969253 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":2}
I20260811 12:46:23.970840 9426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:23.963827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:38947 (local address 127.7.111.67:48289)
0811 12:46:23.965468 (+ 1641us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:23.965490 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:23.965591 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:23.965860 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:23.965862 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:23.965877 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:23.966348 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:23.966364 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.967348 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.967354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:23.968867 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:23.968880 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.970312 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.970317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:23.970431 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:23.970454 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 12:46:23.970527 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 12:46:23.970607 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1501,"thread_start_us":1247,"threads_started":1}
I20260811 12:46:23.970583 8830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a03397e30ca347468ab7add66e04d4c6" candidate_uuid: "48a908c7abe54c04b2de55cd36b1d8fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7db60368d4074410875e8a7b7dbc958f" is_pre_election: true
I20260811 12:46:23.971026 9405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:23.963675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:48289 (local address 127.7.111.66:38947)
0811 12:46:23.963836 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:23.963842 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:23.965569 (+ 1727us) server_negotiation.cc:407] Connection header received
0811 12:46:23.965695 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:23.965699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:23.965765 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:23.965849 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:23.966513 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.967203 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.969021 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:23.970136 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:23.970303 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:23.970375 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:23.970571 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 12:46:23.970697 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 12:46:23.971210 8830 raft_consensus.cc:2454] T a03397e30ca347468ab7add66e04d4c6 P 7db60368d4074410875e8a7b7dbc958f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48a908c7abe54c04b2de55cd36b1d8fc in term 0.
I20260811 12:46:23.971906 9195 leader_election.cc:302] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48a908c7abe54c04b2de55cd36b1d8fc, 7db60368d4074410875e8a7b7dbc958f; no voters:
I20260811 12:46:23.971992 9068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a03397e30ca347468ab7add66e04d4c6" candidate_uuid: "48a908c7abe54c04b2de55cd36b1d8fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af793fda75cf4ee3afc59e2db5b41aeb" is_pre_election: true
I20260811 12:46:23.972263 9424 raft_consensus.cc:2781] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:46:23.972455 9424 raft_consensus.cc:482] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:46:23.972575 9424 raft_consensus.cc:3037] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:23.972551 9068 raft_consensus.cc:2454] T a03397e30ca347468ab7add66e04d4c6 P af793fda75cf4ee3afc59e2db5b41aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48a908c7abe54c04b2de55cd36b1d8fc in term 0.
I20260811 12:46:23.975168 9424 raft_consensus.cc:504] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af793fda75cf4ee3afc59e2db5b41aeb" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 38947 } } peers { permanent_uuid: "7db60368d4074410875e8a7b7dbc958f" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 46721 } } peers { permanent_uuid: "48a908c7abe54c04b2de55cd36b1d8fc" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 46157 } }
I20260811 12:46:23.975831 9424 leader_election.cc:288] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc [CANDIDATE]: Term 1 election: Requested vote from peers af793fda75cf4ee3afc59e2db5b41aeb (127.7.111.66:38947), 7db60368d4074410875e8a7b7dbc958f (127.7.111.65:46721)
I20260811 12:46:23.976214 9068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a03397e30ca347468ab7add66e04d4c6" candidate_uuid: "48a908c7abe54c04b2de55cd36b1d8fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af793fda75cf4ee3afc59e2db5b41aeb"
I20260811 12:46:23.976322 8830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a03397e30ca347468ab7add66e04d4c6" candidate_uuid: "48a908c7abe54c04b2de55cd36b1d8fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7db60368d4074410875e8a7b7dbc958f"
I20260811 12:46:23.976423 9068 raft_consensus.cc:3037] T a03397e30ca347468ab7add66e04d4c6 P af793fda75cf4ee3afc59e2db5b41aeb [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:23.976521 8830 raft_consensus.cc:3037] T a03397e30ca347468ab7add66e04d4c6 P 7db60368d4074410875e8a7b7dbc958f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:23.979928 8830 raft_consensus.cc:2454] T a03397e30ca347468ab7add66e04d4c6 P 7db60368d4074410875e8a7b7dbc958f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48a908c7abe54c04b2de55cd36b1d8fc in term 1.
I20260811 12:46:23.979977 9068 raft_consensus.cc:2454] T a03397e30ca347468ab7add66e04d4c6 P af793fda75cf4ee3afc59e2db5b41aeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48a908c7abe54c04b2de55cd36b1d8fc in term 1.
I20260811 12:46:23.980311 9195 leader_election.cc:302] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48a908c7abe54c04b2de55cd36b1d8fc, 7db60368d4074410875e8a7b7dbc958f; no voters:
I20260811 12:46:23.980582 9424 raft_consensus.cc:2781] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:46:23.981065 9424 raft_consensus.cc:686] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc [term 1 LEADER]: Becoming Leader. State: Replica: 48a908c7abe54c04b2de55cd36b1d8fc, State: Running, Role: LEADER
I20260811 12:46:23.981640 9424 consensus_queue.cc:237] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af793fda75cf4ee3afc59e2db5b41aeb" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 38947 } } peers { permanent_uuid: "7db60368d4074410875e8a7b7dbc958f" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 46721 } } peers { permanent_uuid: "48a908c7abe54c04b2de55cd36b1d8fc" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 46157 } }
I20260811 12:46:23.986177 8591 catalog_manager.cc:5495] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 48a908c7abe54c04b2de55cd36b1d8fc (127.7.111.67). New cstate: current_term: 1 leader_uuid: "48a908c7abe54c04b2de55cd36b1d8fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af793fda75cf4ee3afc59e2db5b41aeb" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 38947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7db60368d4074410875e8a7b7dbc958f" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 46721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48a908c7abe54c04b2de55cd36b1d8fc" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 46157 } health_report { overall_health: HEALTHY } } }
Aug 11 12:46:24 dist-test-slave-f0tn krb5kdc[8509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.65@KRBTEST.COM
I20260811 12:46:24.016829 9407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:24.002212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51898 (local address 127.7.111.65:46721)
0811 12:46:24.002398 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:24.002403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:24.003051 (+ 648us) server_negotiation.cc:407] Connection header received
0811 12:46:24.003186 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:24.003190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:24.003265 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:24.003345 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:24.004834 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:24.005677 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:24.006886 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:24.007381 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:24.007477 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:24.011301 (+ 3824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:24.011316 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:24.011333 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:24.011373 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:24.014311 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:24.014953 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:24.014964 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:24.014967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:24.015147 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:24.015444 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:24.015448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:24.015450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:24.015861 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:24.016189 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:24.016352 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 12:46:24.016517 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 12:46:24.028716 8925 tablet.cc:2369] T a03397e30ca347468ab7add66e04d4c6 P 7db60368d4074410875e8a7b7dbc958f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 12:46:24 dist-test-slave-f0tn krb5kdc[8509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452384, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260811 12:46:24.054351 9160 tablet.cc:2369] T a03397e30ca347468ab7add66e04d4c6 P af793fda75cf4ee3afc59e2db5b41aeb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:46:24.057977 9374 tablet.cc:2369] T a03397e30ca347468ab7add66e04d4c6 P 48a908c7abe54c04b2de55cd36b1d8fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 12:46:24 dist-test-slave-f0tn krb5kdc[8509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452384, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.111.65@KRBTEST.COM
I20260811 12:46:24.066593 9407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:24.052725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51908 (local address 127.7.111.65:46721)
0811 12:46:24.052943 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:24.052950 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:24.053197 (+ 247us) server_negotiation.cc:407] Connection header received
0811 12:46:24.053410 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:24.053417 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:24.053487 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:24.053575 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:24.055036 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:24.055829 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:24.057154 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:24.057724 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:24.057860 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:24.061705 (+ 3845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:24.061710 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:24.061714 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:24.061760 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:24.064419 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:24.064993 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:24.064996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:24.064999 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:24.065121 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:24.065472 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:24.065475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:24.065476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:24.065744 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:24.065988 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:24.066139 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 12:46:24.066314 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 12:46:24 dist-test-slave-f0tn krb5kdc[8509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 12:46:24 dist-test-slave-f0tn krb5kdc[8509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.65@KRBTEST.COM
I20260811 12:46:24.117805 9407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:24.102777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51914 (local address 127.7.111.65:46721)
0811 12:46:24.102962 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:24.102970 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:24.103393 (+ 423us) server_negotiation.cc:407] Connection header received
0811 12:46:24.103589 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:24.103593 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:24.103663 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:24.103761 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:24.105258 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:24.106049 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:24.107413 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:24.107969 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:24.108088 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:24.112206 (+ 4118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:24.112211 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:24.112214 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:24.112261 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:24.115102 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:24.115803 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:24.115807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:24.115809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:24.115938 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:24.116325 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:24.116328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:24.116330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:24.116698 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:24.116891 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:24.117121 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 12:46:24.117361 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 12:46:24.119655 8767 tablet_service.cc:2908] Rejecting scan request for tablet a03397e30ca347468ab7add66e04d4c6: Uninitialized: clean time has not yet been initialized
I20260811 12:46:24.123124 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 8691
I20260811 12:46:24.136540 9407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:24.124067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51924 (local address 127.7.111.65:46721)
0811 12:46:24.124260 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:24.124267 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:24.124578 (+ 311us) server_negotiation.cc:407] Connection header received
0811 12:46:24.124708 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:24.124711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:24.124790 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:24.124870 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:24.126099 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:24.126788 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:24.128101 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:24.128542 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:24.128680 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:24.131133 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:24.131139 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:24.131142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:24.131180 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:24.134102 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:24.134842 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:24.134846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:24.134849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:24.134989 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:24.135273 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:24.135276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:24.135278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:24.135546 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:24.135798 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:24.135991 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 12:46:24.136189 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:46:24.137277 8916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:24.228581 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 8691
I20260811 12:46:24.255591 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 8938
Aug 11 12:46:24 dist-test-slave-f0tn krb5kdc[8509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.66@KRBTEST.COM
I20260811 12:46:24.269591 9405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:24.256373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59678 (local address 127.7.111.66:38947)
0811 12:46:24.256572 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:24.256577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:24.256597 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:46:24.256723 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:24.256727 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:24.256804 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:24.256884 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:24.258188 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:24.258934 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:24.260108 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:24.260498 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:24.260586 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:24.264382 (+ 3796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:24.264399 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:24.264417 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:24.264451 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:24.267089 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:24.267667 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:24.267683 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:24.267685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:24.267811 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:24.268076 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:24.268079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:24.268081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:24.268513 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:24.268820 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:24.268969 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 12:46:24.269135 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:46:24.270247 9148 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:24.357360 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 8938
I20260811 12:46:24.380263 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 9168
Aug 11 12:46:24 dist-test-slave-f0tn krb5kdc[8509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.67@KRBTEST.COM
I20260811 12:46:24.394194 9409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:24.381067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41840 (local address 127.7.111.67:46157)
0811 12:46:24.381260 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:24.381265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:24.381284 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:46:24.381340 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:24.381345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:24.381452 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:24.381534 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:24.382692 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:24.383417 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:24.384681 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:24.385067 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:24.385153 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:24.388997 (+ 3844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:24.389015 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:24.389034 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:24.389068 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:24.391575 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:24.392196 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:24.392203 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:24.392206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:24.392337 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:24.392662 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:24.392665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:24.392667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:24.393025 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:24.393239 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:24.393677 (+ 438us) server_negotiation.cc:299] Negotiation successful
0811 12:46:24.393876 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:46:24.394881 9365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:24.485419 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 9168
I20260811 12:46:24.512945 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 8531
I20260811 12:46:24.514379 8634 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:24.625608 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 8531
2026-08-11T12:46:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4299 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:46:24 dist-test-slave-f0tn krb5kdc[9455](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:46:24 dist-test-slave-f0tn krb5kdc[9455](info): set up 2 sockets
Aug 11 12:46:24 dist-test-slave-f0tn krb5kdc[9455](info): commencing operation
krb5kdc: starting...
W20260811 12:46:25.836731 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.010s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:46:25 dist-test-slave-f0tn krb5kdc[9455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:46:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:46:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:46:26.038363 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:37631
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:45345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:37631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:26.357968 9480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:26.358263 9480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:26.358366 9480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:26.365902 9480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:46:26.366010 9480 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 12:46:26.366063 9480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:26.366122 9480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:26.366169 9480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:46:26.366235 9480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:46:26.376206 9480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:45345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:37631
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:37631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:26.377521 9480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:26.378808 9480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:26.390023 9486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:26.389950 9495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:26.389997 9492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:26.391134 9480 server_base.cc:1034] running on GCE node
I20260811 12:46:26.391865 9480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:26.393267 9480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:26.394483 9480 hybrid_clock.cc:648] HybridClock initialized: now 1786452386394414 us; error 43 us; skew 500 ppm
I20260811 12:46:26.394826 9480 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:26 dist-test-slave-f0tn krb5kdc[9455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452386, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:26.397949 9480 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:46:26.399369 9480 webserver.cc:460] Webserver started at http://127.7.111.126:33761/ using document root <none> and password file <none>
I20260811 12:46:26.400096 9480 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:26.400198 9480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:26.400502 9480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:26.403496 9480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "60cc079943e74141ac1e649304bdeeee"
format_stamp: "Formatted at 2026-08-11 12:46:26 on dist-test-slave-f0tn"
server_key: "e47054c2f88d2d90b341fd0df1f00f70"
server_key_iv: "fddf2afcb55eb327514ef2167c9cf00d"
server_key_version: "encryptionkey@0"
I20260811 12:46:26.404304 9480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "60cc079943e74141ac1e649304bdeeee"
format_stamp: "Formatted at 2026-08-11 12:46:26 on dist-test-slave-f0tn"
server_key: "e47054c2f88d2d90b341fd0df1f00f70"
server_key_iv: "fddf2afcb55eb327514ef2167c9cf00d"
server_key_version: "encryptionkey@0"
I20260811 12:46:26.409271 9480 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:46:26.412596 9504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:26.413968 9480 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:46:26.414142 9480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "60cc079943e74141ac1e649304bdeeee"
format_stamp: "Formatted at 2026-08-11 12:46:26 on dist-test-slave-f0tn"
server_key: "e47054c2f88d2d90b341fd0df1f00f70"
server_key_iv: "fddf2afcb55eb327514ef2167c9cf00d"
server_key_version: "encryptionkey@0"
I20260811 12:46:26.414359 9480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:26.443189 9480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:26.447171 9480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:26.447417 9480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:26.467492 9480 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:37631
I20260811 12:46:26.467512 9625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:37631 every 8 connection(s)
I20260811 12:46:26.468840 9480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:46:26.472579 9626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:26.475682 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 9480
I20260811 12:46:26.475915 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:46:26.476545 7613 external_mini_cluster.cc:1428] Setting key ce5a7ee8d2a707ba996bd727dbda255a
I20260811 12:46:26.484860 9626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee: Bootstrap starting.
I20260811 12:46:26.488597 9626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:26.489921 9626 log.cc:824] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee: Log is configured to *not* fsync() on all Append() calls
Aug 11 12:46:26 dist-test-slave-f0tn krb5kdc[9455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:26.493953 9626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee: No bootstrap required, opened a new log
I20260811 12:46:26.499118 9629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:26.478473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46460 (local address 127.7.111.126:37631)
0811 12:46:26.479232 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:26.479244 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:26.479284 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:46:26.480883 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:26.480906 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:26.481293 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:26.481677 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:26.483399 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:26.484658 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:26.486306 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:26.486967 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:26.487087 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:26.492026 (+ 4939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:26.492033 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:26.492047 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:26.492085 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:26.495223 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:26.495888 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:26.495898 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:26.495906 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:26.496078 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:26.496456 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:26.496460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:26.496462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:26.496994 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:26.497304 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:26.497707 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 12:46:26.498125 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":345,"threads_started":1}
I20260811 12:46:26.501637 9626 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60cc079943e74141ac1e649304bdeeee" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37631 } }
I20260811 12:46:26.502068 9626 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:26.502180 9626 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60cc079943e74141ac1e649304bdeeee, State: Initialized, Role: FOLLOWER
I20260811 12:46:26.502737 9626 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60cc079943e74141ac1e649304bdeeee" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37631 } }
I20260811 12:46:26.502967 9626 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:46:26.503099 9626 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:46:26.503260 9626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:26.505395 9626 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60cc079943e74141ac1e649304bdeeee" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37631 } }
I20260811 12:46:26.505878 9626 leader_election.cc:302] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60cc079943e74141ac1e649304bdeeee; no voters:
I20260811 12:46:26.506461 9626 leader_election.cc:288] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:46:26.506546 9632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:46:26.507529 9632 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee [term 1 LEADER]: Becoming Leader. State: Replica: 60cc079943e74141ac1e649304bdeeee, State: Running, Role: LEADER
I20260811 12:46:26.507983 9632 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60cc079943e74141ac1e649304bdeeee" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37631 } }
I20260811 12:46:26.508494 9626 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:46:26.511308 9633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60cc079943e74141ac1e649304bdeeee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60cc079943e74141ac1e649304bdeeee" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37631 } } }
I20260811 12:46:26.511667 9633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee [sys.catalog]: This master's current role is: LEADER
I20260811 12:46:26.511897 9634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60cc079943e74141ac1e649304bdeeee. Latest consensus state: current_term: 1 leader_uuid: "60cc079943e74141ac1e649304bdeeee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60cc079943e74141ac1e649304bdeeee" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37631 } } }
I20260811 12:46:26.512166 9634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee [sys.catalog]: This master's current role is: LEADER
I20260811 12:46:26.526057 9641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:46:26.531618 9641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:46:26.540575 9641 catalog_manager.cc:1347] Generated new cluster ID: b8a7d6b14c9045cb83c1a45f972fe414
I20260811 12:46:26.540699 9641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:46:26.549017 9641 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:46:26.550282 9641 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:46:26.555984 9641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee: Generated new TSK 0
I20260811 12:46:26.556713 9641 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.111.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
I20260811 12:46:26.609819 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:37631
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:45345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:26.942962 9667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:26.943250 9667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:26.943400 9667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:26.950636 9667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:26.950748 9667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:26.950875 9667 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:46:26.950994 9667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:46:26.960522 9667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:45345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.111.126:37631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:26.961934 9667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:26.963286 9667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:26.975742 9672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:26.975687 9678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:26.975663 9680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:26.976471 9667 server_base.cc:1034] running on GCE node
I20260811 12:46:26.977051 9667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:26.977952 9667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:26.979159 9667 hybrid_clock.cc:648] HybridClock initialized: now 1786452386979071 us; error 63 us; skew 500 ppm
I20260811 12:46:26.979435 9667 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:26 dist-test-slave-f0tn krb5kdc[9455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452386, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:26.982517 9667 init.cc:377] Logged in from keytab as kudu/127.7.111.65@KRBTEST.COM (short username kudu)
I20260811 12:46:26.983814 9667 webserver.cc:460] Webserver started at http://127.7.111.65:45987/ using document root <none> and password file <none>
I20260811 12:46:26.984444 9667 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:26.984525 9667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:26.984790 9667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:26.987577 9667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "aef6c0a5fdc84e6ebc311fe08785cf21"
format_stamp: "Formatted at 2026-08-11 12:46:26 on dist-test-slave-f0tn"
server_key: "ae330966850af106b5d97db2e1b0faaf"
server_key_iv: "a5130de44e509be28306aea73275fa47"
server_key_version: "encryptionkey@0"
I20260811 12:46:26.988306 9667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "aef6c0a5fdc84e6ebc311fe08785cf21"
format_stamp: "Formatted at 2026-08-11 12:46:26 on dist-test-slave-f0tn"
server_key: "ae330966850af106b5d97db2e1b0faaf"
server_key_iv: "a5130de44e509be28306aea73275fa47"
server_key_version: "encryptionkey@0"
I20260811 12:46:26.993088 9667 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 12:46:26.996486 9691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:26.997797 9667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:46:26.997934 9667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "aef6c0a5fdc84e6ebc311fe08785cf21"
format_stamp: "Formatted at 2026-08-11 12:46:26 on dist-test-slave-f0tn"
server_key: "ae330966850af106b5d97db2e1b0faaf"
server_key_iv: "a5130de44e509be28306aea73275fa47"
server_key_version: "encryptionkey@0"
I20260811 12:46:26.998142 9667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:27.011173 9667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:27.015235 9667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:27.015455 9667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:27.018098 9667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:27.018204 9667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:27.018354 9667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:27.018427 9667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:46:27 dist-test-slave-f0tn krb5kdc[9455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452386, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:27.040753 9701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:27.018816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:46311 (local address 127.7.111.126:37631)
0811 12:46:27.021137 (+ 2321us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:27.021144 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:27.021168 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:46:27.022782 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:27.022789 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:27.022881 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:27.022996 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:27.025031 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:27.025972 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:27.028439 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:27.028979 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:27.029101 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:27.034107 (+ 5006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:27.034114 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:27.034119 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:27.034165 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:27.037423 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:27.038264 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:27.038269 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:27.038272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:27.038405 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:27.038984 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:27.038989 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:27.038992 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:27.039349 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:27.039491 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:27.040153 (+ 662us) server_negotiation.cc:299] Negotiation successful
0811 12:46:27.040381 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2198,"thread_start_us":2078,"threads_started":1}
I20260811 12:46:27.041844 9704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:27.019459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:37631 (local address 127.7.111.65:46311)
0811 12:46:27.020426 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:27.020492 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:27.022497 (+ 2005us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:27.023299 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:27.023309 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:27.023930 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:27.024853 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:27.024878 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:27.026183 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:27.026187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:27.028255 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:27.028272 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:27.029238 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:27.029250 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:27.029609 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:27.030483 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:27.030520 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:27.033865 (+ 3345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:27.037570 (+ 3705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:27.037583 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:27.037602 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:27.038134 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:27.038568 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:27.038577 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:27.038581 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:27.038856 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:27.039491 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:27.039503 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:27.039810 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 12:46:27.040045 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 12:46:27.040521 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":377,"threads_started":1}
I20260811 12:46:27.061726 9701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:27.047495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:57429 (local address 127.7.111.126:37631)
0811 12:46:27.047655 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:27.047660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:27.048651 (+ 991us) server_negotiation.cc:407] Connection header received
0811 12:46:27.048858 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:27.048863 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:27.048937 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:27.049031 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:27.050343 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:27.051022 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:27.053117 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:27.053701 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:27.053833 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:27.055978 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:27.055983 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:27.055986 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:27.056016 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:27.058750 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:27.059555 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:27.059560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:27.059563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:27.059719 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:27.060165 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:27.060171 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:27.060175 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:27.060534 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:27.060687 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:27.061155 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 12:46:27.061367 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:46:27.061887 9805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:27.047645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:37631 (local address 127.7.111.65:57429)
0811 12:46:27.048536 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:27.048565 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:27.048746 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:27.049083 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:27.049089 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:27.049717 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:27.050178 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:27.050196 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:27.051137 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:27.051142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:27.052957 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:27.052976 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:27.053930 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:27.053939 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:27.054133 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:27.054726 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:27.054749 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:27.055815 (+ 1066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:27.058870 (+ 3055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:27.058876 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:27.058881 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:27.059447 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:27.059836 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:27.059841 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:27.059845 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:27.060033 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:27.060663 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:27.060667 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:27.060889 (+ 222us) client_negotiation.cc:769] Sending connection context
0811 12:46:27.061038 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 12:46:27.061408 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"thread_start_us":542,"threads_started":1}
I20260811 12:46:27.069414 9667 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:43759
I20260811 12:46:27.069466 9903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:43759 every 8 connection(s)
I20260811 12:46:27.070565 9667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:46:27.072360 9905 heartbeater.cc:347] Connected to a master server at 127.7.111.126:37631
I20260811 12:46:27.072549 9905 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:27.073139 9905 heartbeater.cc:511] Master 127.7.111.126:37631 requested a full tablet report, sending...
I20260811 12:46:27.074998 9534 ts_manager.cc:194] Registered new tserver with Master: aef6c0a5fdc84e6ebc311fe08785cf21 (127.7.111.65:43759)
I20260811 12:46:27.076431 9534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:46311
I20260811 12:46:27.080219 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 9667
I20260811 12:46:27.080410 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:46:27.081022 7613 external_mini_cluster.cc:1428] Setting key 8419234caf20db2c9ff35798cb9ad085
WARNING: no policy specified for kudu/127.7.111.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
I20260811 12:46:27.121675 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:37631
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:45345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:27.448431 9909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:27.448700 9909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:27.448828 9909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:27.455865 9909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:27.455971 9909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:27.456096 9909 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:46:27.456202 9909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:46:27.465862 9909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:45345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.111.126:37631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:27.467296 9909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:27.469075 9909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:27.483953 9918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:27.484164 9920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:27.484570 9915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:27.484594 9909 server_base.cc:1034] running on GCE node
I20260811 12:46:27.485363 9909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:27.486442 9909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:27.487685 9909 hybrid_clock.cc:648] HybridClock initialized: now 1786452387487601 us; error 51 us; skew 500 ppm
I20260811 12:46:27.487974 9909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:27 dist-test-slave-f0tn krb5kdc[9455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452387, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:27.491019 9909 init.cc:377] Logged in from keytab as kudu/127.7.111.66@KRBTEST.COM (short username kudu)
I20260811 12:46:27.492455 9909 webserver.cc:460] Webserver started at http://127.7.111.66:44575/ using document root <none> and password file <none>
I20260811 12:46:27.493304 9909 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:27.493463 9909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:27.493844 9909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:27.497309 9909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "5b61ba89b1524affa03175e80409de8d"
format_stamp: "Formatted at 2026-08-11 12:46:27 on dist-test-slave-f0tn"
server_key: "a6d321c3e50bf96a9c6849d47da6ccab"
server_key_iv: "2d4c4168585621358d7231f557aa4913"
server_key_version: "encryptionkey@0"
I20260811 12:46:27.498224 9909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "5b61ba89b1524affa03175e80409de8d"
format_stamp: "Formatted at 2026-08-11 12:46:27 on dist-test-slave-f0tn"
server_key: "a6d321c3e50bf96a9c6849d47da6ccab"
server_key_iv: "2d4c4168585621358d7231f557aa4913"
server_key_version: "encryptionkey@0"
I20260811 12:46:27.503413 9909 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:46:27.506852 9933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:27.508248 9909 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:46:27.508396 9909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "5b61ba89b1524affa03175e80409de8d"
format_stamp: "Formatted at 2026-08-11 12:46:27 on dist-test-slave-f0tn"
server_key: "a6d321c3e50bf96a9c6849d47da6ccab"
server_key_iv: "2d4c4168585621358d7231f557aa4913"
server_key_version: "encryptionkey@0"
I20260811 12:46:27.508596 9909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:27.528081 9909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:27.532020 9909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:27.532253 9909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:27.534950 9909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:27.535032 9909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:27.535169 9909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:27.535223 9909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:46:27 dist-test-slave-f0tn krb5kdc[9455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452387, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:27.559784 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:27.536215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:37631 (local address 127.7.111.66:48889)
0811 12:46:27.537122 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:27.537178 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:27.539100 (+ 1922us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:27.539841 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:27.539849 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:27.540581 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:27.541472 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:27.541495 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:27.543115 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:27.543121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:27.545241 (+ 2120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:27.545261 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:27.546351 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:27.546358 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:27.546693 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:27.547517 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:27.547550 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:27.550969 (+ 3419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:27.554672 (+ 3703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:27.554682 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:27.554697 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:27.555292 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:27.555757 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:27.555763 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:27.555766 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:27.556097 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:27.556878 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:27.556886 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:27.557236 (+ 350us) client_negotiation.cc:769] Sending connection context
0811 12:46:27.557640 (+ 404us) client_negotiation.cc:240] Negotiation successful
0811 12:46:27.558168 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":388,"threads_started":1}
I20260811 12:46:27.560724 9701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:27.535639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:48889 (local address 127.7.111.126:37631)
0811 12:46:27.535836 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:27.535842 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:27.537322 (+ 1480us) server_negotiation.cc:407] Connection header received
0811 12:46:27.539400 (+ 2078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:27.539405 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:27.539485 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:27.539573 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:27.541739 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:27.542900 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:27.545473 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:27.546099 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:27.546224 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:27.551259 (+ 5035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:27.551267 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:27.551271 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:27.551314 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:27.554510 (+ 3196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:27.555421 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:27.555426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:27.555430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:27.555616 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:27.556225 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:27.556230 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:27.556233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:27.556666 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:27.556811 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:27.560163 (+ 3352us) server_negotiation.cc:299] Negotiation successful
0811 12:46:27.560387 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:46:27.573081 9909 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:37467
I20260811 12:46:27.573168 10121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:37467 every 8 connection(s)
I20260811 12:46:27.574499 9909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:46:27.576728 10122 heartbeater.cc:347] Connected to a master server at 127.7.111.126:37631
I20260811 12:46:27.576943 10122 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:27.577626 10122 heartbeater.cc:511] Master 127.7.111.126:37631 requested a full tablet report, sending...
I20260811 12:46:27.578794 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 9909
I20260811 12:46:27.578956 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:46:27.579150 9534 ts_manager.cc:194] Registered new tserver with Master: 5b61ba89b1524affa03175e80409de8d (127.7.111.66:37467)
I20260811 12:46:27.579355 7613 external_mini_cluster.cc:1428] Setting key 8cf90be9cf21d340b64263fe578ce681
I20260811 12:46:27.579982 9534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:48889
I20260811 12:46:27.582051 9701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:27.567234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:45079 (local address 127.7.111.126:37631)
0811 12:46:27.567382 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:27.567388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:27.568141 (+ 753us) server_negotiation.cc:407] Connection header received
0811 12:46:27.568380 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:27.568386 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:27.568507 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:27.568577 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:27.569994 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:27.570980 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:27.572409 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:27.572848 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:27.572957 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:27.574844 (+ 1887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:27.574849 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:27.574852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:27.574882 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:27.577758 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:27.579852 (+ 2094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:27.579858 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:27.579861 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:27.580039 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:27.580453 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:27.580458 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:27.580460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:27.580886 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:27.581034 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:27.581501 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 12:46:27.581731 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 12:46:27.582239 10101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:27.567452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:37631 (local address 127.7.111.66:45079)
0811 12:46:27.568040 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:27.568055 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:27.568222 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:27.568785 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:27.568788 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:27.569401 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:27.569854 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:27.569869 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:27.571136 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:27.571142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:27.572217 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:27.572236 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:27.573042 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:27.573049 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:27.573253 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:27.573862 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:27.573878 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:27.574666 (+ 788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:27.577877 (+ 3211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:27.577882 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:27.577885 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:27.578391 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:27.580181 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:27.580186 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:27.580189 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:27.580351 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:27.581005 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:27.581010 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:27.581239 (+ 229us) client_negotiation.cc:769] Sending connection context
0811 12:46:27.581371 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 12:46:27.581798 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"mutex_wait_us":67,"thread_start_us":317,"threads_started":1}
WARNING: no policy specified for kudu/127.7.111.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
I20260811 12:46:27.626748 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:37631
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:45345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:27.951390 10131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:27.951651 10131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:27.951800 10131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:27.959311 10131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:27.959441 10131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:27.959579 10131 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:46:27.959697 10131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:46:27.969802 10131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:45345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.111.126:37631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:27.971068 10131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:27.972501 10131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:27.985746 10139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:27.985802 10138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:27.985845 10141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:27.986641 10131 server_base.cc:1034] running on GCE node
I20260811 12:46:27.987250 10131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:27.988152 10131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:27.989420 10131 hybrid_clock.cc:648] HybridClock initialized: now 1786452387989312 us; error 38 us; skew 500 ppm
I20260811 12:46:27.989715 10131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:27 dist-test-slave-f0tn krb5kdc[9455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452387, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:27.992703 10131 init.cc:377] Logged in from keytab as kudu/127.7.111.67@KRBTEST.COM (short username kudu)
I20260811 12:46:27.994166 10131 webserver.cc:460] Webserver started at http://127.7.111.67:38739/ using document root <none> and password file <none>
I20260811 12:46:27.994859 10131 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:27.994984 10131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:27.995352 10131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:27.998443 10131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "47b60b3751854c1d87af5a75cae1bb2b"
format_stamp: "Formatted at 2026-08-11 12:46:27 on dist-test-slave-f0tn"
server_key: "10dd38c5e49638455085081381b22bac"
server_key_iv: "1ad9e35d3eca686f56ff7d3c833f5b3f"
server_key_version: "encryptionkey@0"
I20260811 12:46:27.999184 10131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "47b60b3751854c1d87af5a75cae1bb2b"
format_stamp: "Formatted at 2026-08-11 12:46:27 on dist-test-slave-f0tn"
server_key: "10dd38c5e49638455085081381b22bac"
server_key_iv: "1ad9e35d3eca686f56ff7d3c833f5b3f"
server_key_version: "encryptionkey@0"
I20260811 12:46:28.004213 10131 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:46:28.007611 10158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:28.008909 10131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:46:28.009060 10131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "47b60b3751854c1d87af5a75cae1bb2b"
format_stamp: "Formatted at 2026-08-11 12:46:27 on dist-test-slave-f0tn"
server_key: "10dd38c5e49638455085081381b22bac"
server_key_iv: "1ad9e35d3eca686f56ff7d3c833f5b3f"
server_key_version: "encryptionkey@0"
I20260811 12:46:28.009263 10131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:28.023319 10131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:28.027285 10131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:28.027544 10131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:28.030237 10131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:28.030397 10131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 12:46:28.030547 10131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:28.030611 10131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:46:28 dist-test-slave-f0tn krb5kdc[9455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452387, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:28.052273 9701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:28.030551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:54607 (local address 127.7.111.126:37631)
0811 12:46:28.030739 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:28.030745 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:28.032120 (+ 1375us) server_negotiation.cc:407] Connection header received
0811 12:46:28.034179 (+ 2059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:28.034186 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:28.034274 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:28.034402 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:28.036516 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.037725 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.040009 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.040421 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.040540 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:28.045361 (+ 4821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:28.045368 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:28.045372 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:28.045445 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:28.048850 (+ 3405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:28.049775 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:28.049782 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:28.049786 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:28.049962 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:28.050368 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:28.050372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:28.050376 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:28.050767 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:28.050920 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:28.051578 (+ 658us) server_negotiation.cc:299] Negotiation successful
0811 12:46:28.051857 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 12:46:28.052913 10178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:28.031104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:37631 (local address 127.7.111.67:54607)
0811 12:46:28.031935 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:28.031984 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:28.033902 (+ 1918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:28.034786 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:28.034797 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:28.035508 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:28.036306 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:28.036334 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.037895 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.037904 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:28.039867 (+ 1963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:28.039883 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.040683 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.040692 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:28.040921 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:28.041639 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:28.041667 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:28.045112 (+ 3445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:28.048995 (+ 3883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:28.049004 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:28.049025 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:28.049636 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:28.050070 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:28.050073 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:28.050075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:28.050266 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:28.050877 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:28.050882 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:28.051174 (+ 292us) client_negotiation.cc:769] Sending connection context
0811 12:46:28.051417 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 12:46:28.051781 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":333,"threads_started":1}
I20260811 12:46:28.072009 9701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:28.059430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:39775 (local address 127.7.111.126:37631)
0811 12:46:28.059610 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:28.059617 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:28.060299 (+ 682us) server_negotiation.cc:407] Connection header received
0811 12:46:28.060464 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:28.060467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:28.060523 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:28.060589 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:28.061894 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.062871 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.064896 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.065284 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.065413 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:28.067068 (+ 1655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:28.067073 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:28.067076 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:28.067106 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:28.069616 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:28.070230 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:28.070234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:28.070236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:28.070353 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:28.070705 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:28.070709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:28.070713 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:28.070944 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:28.071070 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:28.071478 (+ 408us) server_negotiation.cc:299] Negotiation successful
0811 12:46:28.071677 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:46:28.072110 10316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:28.059550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:37631 (local address 127.7.111.67:39775)
0811 12:46:28.060186 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:28.060208 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:28.060352 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:28.060631 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:28.060637 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:28.061232 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:28.061659 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:28.061684 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.063018 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.063024 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:28.064752 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:28.064769 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.065519 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.065527 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:28.065668 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:28.066193 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:28.066212 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:28.066900 (+ 688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:28.069736 (+ 2836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:28.069741 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:28.069745 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:28.070117 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:28.070455 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:28.070460 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:28.070464 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:28.070604 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:28.071054 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:28.071058 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:28.071233 (+ 175us) client_negotiation.cc:769] Sending connection context
0811 12:46:28.071354 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 12:46:28.071647 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":318,"threads_started":1}
I20260811 12:46:28.078920 9905 heartbeater.cc:503] Master 127.7.111.126:37631 was elected leader, sending a full tablet report...
I20260811 12:46:28.083240 10131 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:45345
I20260811 12:46:28.083272 10369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:45345 every 8 connection(s)
I20260811 12:46:28.084378 10131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:46:28.085492 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 10131
I20260811 12:46:28.085711 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:46:28.086233 7613 external_mini_cluster.cc:1428] Setting key 3af712efcebc126f7aaf2239ab980186
I20260811 12:46:28.086400 10370 heartbeater.cc:347] Connected to a master server at 127.7.111.126:37631
I20260811 12:46:28.086606 10370 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:28.087371 10370 heartbeater.cc:511] Master 127.7.111.126:37631 requested a full tablet report, sending...
I20260811 12:46:28.088703 9534 ts_manager.cc:194] Registered new tserver with Master: 47b60b3751854c1d87af5a75cae1bb2b (127.7.111.67:45345)
I20260811 12:46:28.089567 9534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:54607
I20260811 12:46:28.100939 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 12:46:28 dist-test-slave-f0tn krb5kdc[9455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452388, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 12:46:28 dist-test-slave-f0tn krb5kdc[9455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452388, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:28.151624 9701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:28.137836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46476 (local address 127.7.111.126:37631)
0811 12:46:28.138047 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:28.138054 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:28.138381 (+ 327us) server_negotiation.cc:407] Connection header received
0811 12:46:28.138566 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:28.138570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:28.138635 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:28.138721 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:28.140114 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.140880 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.142215 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.142641 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.142756 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:28.146703 (+ 3947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:28.146708 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:28.146711 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:28.146747 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:28.149060 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:28.149753 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:28.149759 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:28.149762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:28.149889 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:28.150259 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:28.150262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:28.150264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:28.150551 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:28.150788 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:28.151080 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 12:46:28.151314 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:46:28.155583 9534 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:46:28.162734 9534 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:46:28.198072 10395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:28.174691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:43759 (local address 127.0.0.1:34804)
0811 12:46:28.187552 (+ 12861us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:28.187597 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:28.188790 (+ 1193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:28.189656 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:28.189664 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:28.189696 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:28.190118 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:28.190133 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.192367 (+ 2234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.192373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:28.194964 (+ 2591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:28.194985 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.197043 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.197050 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:28.197241 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:28.197295 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 12:46:28.197454 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 12:46:28.197651 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12554,"thread_start_us":13233,"threads_started":1}
I20260811 12:46:28.198069 10400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:28.175320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34804 (local address 127.7.111.65:43759)
0811 12:46:28.188554 (+ 13234us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:28.188563 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:28.189003 (+ 440us) server_negotiation.cc:407] Connection header received
0811 12:46:28.189119 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:28.189127 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:28.189414 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:28.190381 (+ 967us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:28.190607 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.192194 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.195239 (+ 3045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.196790 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.196966 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:28.197183 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:28.197595 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 12:46:28.197772 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13063,"thread_start_us":13812,"threads_started":1}
I20260811 12:46:28.199935 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:28.174806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:45345 (local address 127.0.0.1:41314)
0811 12:46:28.189487 (+ 14681us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:28.189504 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:28.189620 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:28.190330 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:28.190335 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:28.190364 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:28.191900 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:28.191917 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.194369 (+ 2452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.194374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:28.197340 (+ 2966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:28.197351 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.199145 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.199151 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:28.199337 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:28.199370 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:46:28.199460 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 12:46:28.199581 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14542,"thread_start_us":14388,"threads_started":1}
I20260811 12:46:28.199990 10397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:28.174954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41314 (local address 127.7.111.67:45345)
0811 12:46:28.189121 (+ 14167us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:28.189128 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:28.189744 (+ 616us) server_negotiation.cc:407] Connection header received
0811 12:46:28.189851 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:28.189860 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:28.190100 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:28.192107 (+ 2007us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:28.192473 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.194203 (+ 1730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.197533 (+ 3330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.198941 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.199070 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:28.199301 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:28.199525 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 12:46:28.199683 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13996,"thread_start_us":13804,"threads_started":1}
I20260811 12:46:28.202929 9779 tablet_service.cc:1467] Processing CreateTablet for tablet bdfbacd29fd549778d21c73f118cecbf (DEFAULT_TABLE table=test-table [id=64e67b0ec3434f3e9a52a3eb1c3d42e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:28.203909 10401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:28.189615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:37467 (local address 127.0.0.1:42424)
0811 12:46:28.196150 (+ 6535us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:28.196171 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:28.196280 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:28.197572 (+ 1292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:28.197576 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:28.197603 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:28.197999 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:28.198011 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.199832 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.199837 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:28.201479 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:28.201494 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.203165 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.203172 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:28.203345 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:28.203378 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:46:28.203469 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 12:46:28.203592 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6411,"thread_start_us":6723,"threads_started":1}
I20260811 12:46:28.203696 10248 tablet_service.cc:1467] Processing CreateTablet for tablet bdfbacd29fd549778d21c73f118cecbf (DEFAULT_TABLE table=test-table [id=64e67b0ec3434f3e9a52a3eb1c3d42e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:28.204782 9779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdfbacd29fd549778d21c73f118cecbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:28.205008 10248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdfbacd29fd549778d21c73f118cecbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:28.205030 10402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:28.189894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42424 (local address 127.7.111.66:37467)
0811 12:46:28.196954 (+ 7060us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:28.196962 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:28.196990 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:46:28.197096 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:28.197103 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:28.197327 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:28.198249 (+ 922us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:28.198430 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.199695 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.201636 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.202952 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.203093 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:28.203305 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:28.204515 (+ 1210us) server_negotiation.cc:299] Negotiation successful
0811 12:46:28.204764 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6903,"thread_start_us":6788,"threads_started":1}
I20260811 12:46:28.208673 10008 tablet_service.cc:1467] Processing CreateTablet for tablet bdfbacd29fd549778d21c73f118cecbf (DEFAULT_TABLE table=test-table [id=64e67b0ec3434f3e9a52a3eb1c3d42e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:28.210072 10008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdfbacd29fd549778d21c73f118cecbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:28.216560 10403 tablet_bootstrap.cc:492] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b: Bootstrap starting.
I20260811 12:46:28.219445 10404 tablet_bootstrap.cc:492] T bdfbacd29fd549778d21c73f118cecbf P aef6c0a5fdc84e6ebc311fe08785cf21: Bootstrap starting.
I20260811 12:46:28.220166 10403 tablet_bootstrap.cc:654] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:28.221663 10403 log.cc:824] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b: Log is configured to *not* fsync() on all Append() calls
I20260811 12:46:28.223119 10404 tablet_bootstrap.cc:654] T bdfbacd29fd549778d21c73f118cecbf P aef6c0a5fdc84e6ebc311fe08785cf21: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:28.224685 10404 log.cc:824] T bdfbacd29fd549778d21c73f118cecbf P aef6c0a5fdc84e6ebc311fe08785cf21: Log is configured to *not* fsync() on all Append() calls
I20260811 12:46:28.225003 10407 tablet_bootstrap.cc:492] T bdfbacd29fd549778d21c73f118cecbf P 5b61ba89b1524affa03175e80409de8d: Bootstrap starting.
I20260811 12:46:28.225319 10403 tablet_bootstrap.cc:492] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b: No bootstrap required, opened a new log
I20260811 12:46:28.225781 10403 ts_tablet_manager.cc:1397] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 12:46:28.228433 10404 tablet_bootstrap.cc:492] T bdfbacd29fd549778d21c73f118cecbf P aef6c0a5fdc84e6ebc311fe08785cf21: No bootstrap required, opened a new log
I20260811 12:46:28.228587 10407 tablet_bootstrap.cc:654] T bdfbacd29fd549778d21c73f118cecbf P 5b61ba89b1524affa03175e80409de8d: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:28.228734 10404 ts_tablet_manager.cc:1397] T bdfbacd29fd549778d21c73f118cecbf P aef6c0a5fdc84e6ebc311fe08785cf21: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260811 12:46:28.229905 10407 log.cc:824] T bdfbacd29fd549778d21c73f118cecbf P 5b61ba89b1524affa03175e80409de8d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:46:28.232774 10407 tablet_bootstrap.cc:492] T bdfbacd29fd549778d21c73f118cecbf P 5b61ba89b1524affa03175e80409de8d: No bootstrap required, opened a new log
I20260811 12:46:28.233034 10407 ts_tablet_manager.cc:1397] T bdfbacd29fd549778d21c73f118cecbf P 5b61ba89b1524affa03175e80409de8d: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 12:46:28.234189 10403 raft_consensus.cc:348] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } } peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } }
I20260811 12:46:28.234579 10403 raft_consensus.cc:374] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:28.234694 10403 raft_consensus.cc:729] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47b60b3751854c1d87af5a75cae1bb2b, State: Initialized, Role: FOLLOWER
I20260811 12:46:28.235247 10403 consensus_queue.cc:260] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } } peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } }
I20260811 12:46:28.236441 10370 heartbeater.cc:503] Master 127.7.111.126:37631 was elected leader, sending a full tablet report...
I20260811 12:46:28.238368 10403 ts_tablet_manager.cc:1428] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260811 12:46:28.238991 10404 raft_consensus.cc:348] T bdfbacd29fd549778d21c73f118cecbf P aef6c0a5fdc84e6ebc311fe08785cf21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } } peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } }
I20260811 12:46:28.239403 10404 raft_consensus.cc:374] T bdfbacd29fd549778d21c73f118cecbf P aef6c0a5fdc84e6ebc311fe08785cf21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:28.239506 10404 raft_consensus.cc:729] T bdfbacd29fd549778d21c73f118cecbf P aef6c0a5fdc84e6ebc311fe08785cf21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aef6c0a5fdc84e6ebc311fe08785cf21, State: Initialized, Role: FOLLOWER
I20260811 12:46:28.240106 10404 consensus_queue.cc:260] T bdfbacd29fd549778d21c73f118cecbf P aef6c0a5fdc84e6ebc311fe08785cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } } peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } }
I20260811 12:46:28.241912 10404 ts_tablet_manager.cc:1428] T bdfbacd29fd549778d21c73f118cecbf P aef6c0a5fdc84e6ebc311fe08785cf21: Time spent starting tablet: real 0.013s user 0.013s sys 0.002s
I20260811 12:46:28.245429 10407 raft_consensus.cc:348] T bdfbacd29fd549778d21c73f118cecbf P 5b61ba89b1524affa03175e80409de8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } } peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } }
I20260811 12:46:28.245770 10407 raft_consensus.cc:374] T bdfbacd29fd549778d21c73f118cecbf P 5b61ba89b1524affa03175e80409de8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:28.245872 10407 raft_consensus.cc:729] T bdfbacd29fd549778d21c73f118cecbf P 5b61ba89b1524affa03175e80409de8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b61ba89b1524affa03175e80409de8d, State: Initialized, Role: FOLLOWER
I20260811 12:46:28.246376 10407 consensus_queue.cc:260] T bdfbacd29fd549778d21c73f118cecbf P 5b61ba89b1524affa03175e80409de8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } } peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } }
I20260811 12:46:28.247427 10122 heartbeater.cc:503] Master 127.7.111.126:37631 was elected leader, sending a full tablet report...
I20260811 12:46:28.247768 10407 ts_tablet_manager.cc:1428] T bdfbacd29fd549778d21c73f118cecbf P 5b61ba89b1524affa03175e80409de8d: Time spent starting tablet: real 0.015s user 0.009s sys 0.005s
W20260811 12:46:28.322793 9906 tablet.cc:2369] T bdfbacd29fd549778d21c73f118cecbf P aef6c0a5fdc84e6ebc311fe08785cf21: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:46:28.326413 10123 tablet.cc:2369] T bdfbacd29fd549778d21c73f118cecbf P 5b61ba89b1524affa03175e80409de8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:46:28.335975 10374 tablet.cc:2369] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:46:28.346737 10410 raft_consensus.cc:482] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:46:28.347048 10410 raft_consensus.cc:504] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } } peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } }
I20260811 12:46:28.348824 10410 leader_election.cc:288] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aef6c0a5fdc84e6ebc311fe08785cf21 (127.7.111.65:43759), 5b61ba89b1524affa03175e80409de8d (127.7.111.66:37467)
I20260811 12:46:28.354411 10400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:28.349050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:42957 (local address 127.7.111.65:43759)
0811 12:46:28.349256 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:28.349263 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:28.349454 (+ 191us) server_negotiation.cc:407] Connection header received
0811 12:46:28.349693 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:28.349698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:28.349774 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:28.349849 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:28.350268 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.351014 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.352606 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.353518 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.353659 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:28.353734 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:28.354047 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 12:46:28.354174 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:46:28.354434 10178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:28.348955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:43759 (local address 127.7.111.67:42957)
0811 12:46:28.349326 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:28.349347 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:28.349506 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:28.349903 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:28.349907 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:28.349935 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:28.350123 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:28.350132 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.351165 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.351168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:28.352447 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:28.352459 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.353655 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.353661 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:28.353897 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:28.353923 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 12:46:28.354006 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 12:46:28.354132 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":21}
I20260811 12:46:28.355505 10402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:28.349542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:53431 (local address 127.7.111.66:37467)
0811 12:46:28.349761 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:28.349766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:28.350087 (+ 321us) server_negotiation.cc:407] Connection header received
0811 12:46:28.350274 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:28.350278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:28.350338 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:28.350411 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:28.350984 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.351793 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.353560 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.354366 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.354548 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:28.354623 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:28.355174 (+ 551us) server_negotiation.cc:299] Negotiation successful
0811 12:46:28.355275 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 12:46:28.355525 9821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdfbacd29fd549778d21c73f118cecbf" candidate_uuid: "47b60b3751854c1d87af5a75cae1bb2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" is_pre_election: true
I20260811 12:46:28.355890 10413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:28.349145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:37467 (local address 127.7.111.67:53431)
0811 12:46:28.349977 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:28.350000 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:28.350140 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:28.350445 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:28.350449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:28.350474 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:28.350841 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:28.350851 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.351933 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.351937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:28.353416 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:28.353432 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.354879 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.354883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:28.355017 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:28.355049 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:46:28.355150 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 12:46:28.355281 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":1,"thread_start_us":619,"threads_started":1}
I20260811 12:46:28.356128 9821 raft_consensus.cc:2454] T bdfbacd29fd549778d21c73f118cecbf P aef6c0a5fdc84e6ebc311fe08785cf21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47b60b3751854c1d87af5a75cae1bb2b in term 0.
I20260811 12:46:28.356778 10159 leader_election.cc:302] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47b60b3751854c1d87af5a75cae1bb2b, aef6c0a5fdc84e6ebc311fe08785cf21; no voters:
I20260811 12:46:28.356884 10040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdfbacd29fd549778d21c73f118cecbf" candidate_uuid: "47b60b3751854c1d87af5a75cae1bb2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b61ba89b1524affa03175e80409de8d" is_pre_election: true
I20260811 12:46:28.357121 10410 raft_consensus.cc:2781] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:46:28.357326 10410 raft_consensus.cc:482] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:46:28.357409 10040 raft_consensus.cc:2454] T bdfbacd29fd549778d21c73f118cecbf P 5b61ba89b1524affa03175e80409de8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47b60b3751854c1d87af5a75cae1bb2b in term 0.
I20260811 12:46:28.357496 10410 raft_consensus.cc:3037] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:28.359947 10410 raft_consensus.cc:504] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } } peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } }
I20260811 12:46:28.360545 10410 leader_election.cc:288] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b [CANDIDATE]: Term 1 election: Requested vote from peers aef6c0a5fdc84e6ebc311fe08785cf21 (127.7.111.65:43759), 5b61ba89b1524affa03175e80409de8d (127.7.111.66:37467)
I20260811 12:46:28.360914 9821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdfbacd29fd549778d21c73f118cecbf" candidate_uuid: "47b60b3751854c1d87af5a75cae1bb2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21"
I20260811 12:46:28.360970 10040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdfbacd29fd549778d21c73f118cecbf" candidate_uuid: "47b60b3751854c1d87af5a75cae1bb2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b61ba89b1524affa03175e80409de8d"
I20260811 12:46:28.361131 10040 raft_consensus.cc:3037] T bdfbacd29fd549778d21c73f118cecbf P 5b61ba89b1524affa03175e80409de8d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:28.361145 9821 raft_consensus.cc:3037] T bdfbacd29fd549778d21c73f118cecbf P aef6c0a5fdc84e6ebc311fe08785cf21 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:28.363966 9821 raft_consensus.cc:2454] T bdfbacd29fd549778d21c73f118cecbf P aef6c0a5fdc84e6ebc311fe08785cf21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47b60b3751854c1d87af5a75cae1bb2b in term 1.
I20260811 12:46:28.364144 10040 raft_consensus.cc:2454] T bdfbacd29fd549778d21c73f118cecbf P 5b61ba89b1524affa03175e80409de8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47b60b3751854c1d87af5a75cae1bb2b in term 1.
I20260811 12:46:28.364320 10159 leader_election.cc:302] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47b60b3751854c1d87af5a75cae1bb2b, aef6c0a5fdc84e6ebc311fe08785cf21; no voters:
I20260811 12:46:28.364584 10410 raft_consensus.cc:2781] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:46:28.365067 10410 raft_consensus.cc:686] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b [term 1 LEADER]: Becoming Leader. State: Replica: 47b60b3751854c1d87af5a75cae1bb2b, State: Running, Role: LEADER
I20260811 12:46:28.365623 10410 consensus_queue.cc:237] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } } peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } }
I20260811 12:46:28.370334 9534 catalog_manager.cc:5495] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b reported cstate change: term changed from 0 to 1, leader changed from <none> to 47b60b3751854c1d87af5a75cae1bb2b (127.7.111.67). New cstate: current_term: 1 leader_uuid: "47b60b3751854c1d87af5a75cae1bb2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } health_report { overall_health: UNKNOWN } } }
I20260811 12:46:28.430568 10397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:28.425227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41316 (local address 127.7.111.67:45345)
0811 12:46:28.425510 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:28.425518 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:28.425542 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:46:28.425848 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:28.425853 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:28.425951 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:28.426073 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:46:28.426387 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.427141 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.428462 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.428915 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.429007 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:28.429431 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:46:28.429986 (+ 555us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:46:28.430089 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:28.430187 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 12:46:28.430253 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 12:46:28.439239 10040 raft_consensus.cc:1261] T bdfbacd29fd549778d21c73f118cecbf P 5b61ba89b1524affa03175e80409de8d [term 1 FOLLOWER]: Refusing update from remote peer 47b60b3751854c1d87af5a75cae1bb2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:46:28.439244 9821 raft_consensus.cc:1261] T bdfbacd29fd549778d21c73f118cecbf P aef6c0a5fdc84e6ebc311fe08785cf21 [term 1 FOLLOWER]: Refusing update from remote peer 47b60b3751854c1d87af5a75cae1bb2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:46:28.440150 10415 consensus_queue.cc:1035] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:46:28.440488 10410 consensus_queue.cc:1035] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:46:28.454516 10423 mvcc.cc:205] Tried to move back new op lower bound from 7317308983037214720 to 7317308982749310976. Current Snapshot: MvccSnapshot[applied={T|T < 7317308983037214720}]
I20260811 12:46:28.472343 9534 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:46:28.472685 9534 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:46:28.476966 9534 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee: Sending DeleteTablet for 3 replicas of tablet bdfbacd29fd549778d21c73f118cecbf
I20260811 12:46:28.478412 10248 tablet_service.cc:1514] Processing DeleteTablet for tablet bdfbacd29fd549778d21c73f118cecbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:28 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:41314
I20260811 12:46:28.478489 9779 tablet_service.cc:1514] Processing DeleteTablet for tablet bdfbacd29fd549778d21c73f118cecbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:28 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:34804
I20260811 12:46:28.479084 10008 tablet_service.cc:1514] Processing DeleteTablet for tablet bdfbacd29fd549778d21c73f118cecbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:28 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:42424
I20260811 12:46:28.483757 9534 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:46:28.484519 9534 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:46:28.495352 9778 tablet_service.cc:1467] Processing CreateTablet for tablet 99b21241f7304a26a7c07006ca24bdcd (DEFAULT_TABLE table=test-table [id=5dcc1a39e432422c939671dacbad95f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:28.496093 9778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99b21241f7304a26a7c07006ca24bdcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:28.497975 10007 tablet_service.cc:1467] Processing CreateTablet for tablet 99b21241f7304a26a7c07006ca24bdcd (DEFAULT_TABLE table=test-table [id=5dcc1a39e432422c939671dacbad95f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:28.498755 10007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99b21241f7304a26a7c07006ca24bdcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:28.495805 10247 tablet_service.cc:1467] Processing CreateTablet for tablet 99b21241f7304a26a7c07006ca24bdcd (DEFAULT_TABLE table=test-table [id=5dcc1a39e432422c939671dacbad95f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:28.501163 10247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99b21241f7304a26a7c07006ca24bdcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:28.506920 10438 tablet_replica.cc:330] T bdfbacd29fd549778d21c73f118cecbf P 5b61ba89b1524affa03175e80409de8d: stopping tablet replica
I20260811 12:46:28.507444 10438 raft_consensus.cc:2229] T bdfbacd29fd549778d21c73f118cecbf P 5b61ba89b1524affa03175e80409de8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:46:28.508345 10403 tablet_bootstrap.cc:492] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b: Bootstrap starting.
I20260811 12:46:28.509505 10438 raft_consensus.cc:2258] T bdfbacd29fd549778d21c73f118cecbf P 5b61ba89b1524affa03175e80409de8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:46:28.510231 10403 tablet_bootstrap.cc:654] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:28.510941 10407 tablet_bootstrap.cc:492] T 99b21241f7304a26a7c07006ca24bdcd P 5b61ba89b1524affa03175e80409de8d: Bootstrap starting.
I20260811 12:46:28.512452 10438 ts_tablet_manager.cc:1905] T bdfbacd29fd549778d21c73f118cecbf P 5b61ba89b1524affa03175e80409de8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:46:28.512799 10407 tablet_bootstrap.cc:654] T 99b21241f7304a26a7c07006ca24bdcd P 5b61ba89b1524affa03175e80409de8d: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:28.517968 10437 tablet_replica.cc:330] T bdfbacd29fd549778d21c73f118cecbf P aef6c0a5fdc84e6ebc311fe08785cf21: stopping tablet replica
I20260811 12:46:28.518496 10436 tablet_replica.cc:330] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b: stopping tablet replica
I20260811 12:46:28.518481 10437 raft_consensus.cc:2229] T bdfbacd29fd549778d21c73f118cecbf P aef6c0a5fdc84e6ebc311fe08785cf21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:46:28.519217 10436 raft_consensus.cc:2229] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:46:28.519960 10436 raft_consensus.cc:2258] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:46:28.520063 10438 ts_tablet_manager.cc:1918] T bdfbacd29fd549778d21c73f118cecbf P 5b61ba89b1524affa03175e80409de8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:46:28.520234 10438 log.cc:1196] T bdfbacd29fd549778d21c73f118cecbf P 5b61ba89b1524affa03175e80409de8d: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/bdfbacd29fd549778d21c73f118cecbf
I20260811 12:46:28.520792 10438 ts_tablet_manager.cc:1939] T bdfbacd29fd549778d21c73f118cecbf P 5b61ba89b1524affa03175e80409de8d: Deleting consensus metadata
I20260811 12:46:28.522505 10436 ts_tablet_manager.cc:1905] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:46:28.522891 9506 catalog_manager.cc:4841] TS 5b61ba89b1524affa03175e80409de8d (127.7.111.66:37467): tablet bdfbacd29fd549778d21c73f118cecbf (table test-table [id=64e67b0ec3434f3e9a52a3eb1c3d42e7]) successfully deleted
I20260811 12:46:28.524786 10403 tablet_bootstrap.cc:492] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b: No bootstrap required, opened a new log
I20260811 12:46:28.524991 10403 ts_tablet_manager.cc:1397] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.000s
I20260811 12:46:28.525837 10407 tablet_bootstrap.cc:492] T 99b21241f7304a26a7c07006ca24bdcd P 5b61ba89b1524affa03175e80409de8d: No bootstrap required, opened a new log
I20260811 12:46:28.525993 10407 ts_tablet_manager.cc:1397] T 99b21241f7304a26a7c07006ca24bdcd P 5b61ba89b1524affa03175e80409de8d: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.006s
I20260811 12:46:28.526090 10403 raft_consensus.cc:348] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } }
I20260811 12:46:28.526324 10403 raft_consensus.cc:374] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:28.526434 10403 raft_consensus.cc:729] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47b60b3751854c1d87af5a75cae1bb2b, State: Initialized, Role: FOLLOWER
I20260811 12:46:28.526787 10407 raft_consensus.cc:348] T 99b21241f7304a26a7c07006ca24bdcd P 5b61ba89b1524affa03175e80409de8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } }
I20260811 12:46:28.526809 10403 consensus_queue.cc:260] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } }
I20260811 12:46:28.527005 10407 raft_consensus.cc:374] T 99b21241f7304a26a7c07006ca24bdcd P 5b61ba89b1524affa03175e80409de8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:28.527231 10407 raft_consensus.cc:729] T 99b21241f7304a26a7c07006ca24bdcd P 5b61ba89b1524affa03175e80409de8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b61ba89b1524affa03175e80409de8d, State: Initialized, Role: FOLLOWER
I20260811 12:46:28.527387 10403 ts_tablet_manager.cc:1428] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 12:46:28.527653 10404 tablet_bootstrap.cc:492] T 99b21241f7304a26a7c07006ca24bdcd P aef6c0a5fdc84e6ebc311fe08785cf21: Bootstrap starting.
I20260811 12:46:28.527624 10407 consensus_queue.cc:260] T 99b21241f7304a26a7c07006ca24bdcd P 5b61ba89b1524affa03175e80409de8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } }
I20260811 12:46:28.529623 10404 tablet_bootstrap.cc:654] T 99b21241f7304a26a7c07006ca24bdcd P aef6c0a5fdc84e6ebc311fe08785cf21: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:28.530381 10407 ts_tablet_manager.cc:1428] T 99b21241f7304a26a7c07006ca24bdcd P 5b61ba89b1524affa03175e80409de8d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 12:46:28.530539 10437 raft_consensus.cc:2258] T bdfbacd29fd549778d21c73f118cecbf P aef6c0a5fdc84e6ebc311fe08785cf21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:46:28.535329 10437 ts_tablet_manager.cc:1905] T bdfbacd29fd549778d21c73f118cecbf P aef6c0a5fdc84e6ebc311fe08785cf21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:46:28.536053 10436 ts_tablet_manager.cc:1918] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:46:28.536239 10436 log.cc:1196] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/bdfbacd29fd549778d21c73f118cecbf
I20260811 12:46:28.536772 10436 ts_tablet_manager.cc:1939] T bdfbacd29fd549778d21c73f118cecbf P 47b60b3751854c1d87af5a75cae1bb2b: Deleting consensus metadata
I20260811 12:46:28.538482 9506 catalog_manager.cc:4841] TS 47b60b3751854c1d87af5a75cae1bb2b (127.7.111.67:45345): tablet bdfbacd29fd549778d21c73f118cecbf (table test-table [id=64e67b0ec3434f3e9a52a3eb1c3d42e7]) successfully deleted
I20260811 12:46:28.539875 10404 tablet_bootstrap.cc:492] T 99b21241f7304a26a7c07006ca24bdcd P aef6c0a5fdc84e6ebc311fe08785cf21: No bootstrap required, opened a new log
I20260811 12:46:28.540019 10404 ts_tablet_manager.cc:1397] T 99b21241f7304a26a7c07006ca24bdcd P aef6c0a5fdc84e6ebc311fe08785cf21: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.007s
I20260811 12:46:28.540841 10404 raft_consensus.cc:348] T 99b21241f7304a26a7c07006ca24bdcd P aef6c0a5fdc84e6ebc311fe08785cf21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } }
I20260811 12:46:28.541077 10404 raft_consensus.cc:374] T 99b21241f7304a26a7c07006ca24bdcd P aef6c0a5fdc84e6ebc311fe08785cf21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:28.541184 10404 raft_consensus.cc:729] T 99b21241f7304a26a7c07006ca24bdcd P aef6c0a5fdc84e6ebc311fe08785cf21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aef6c0a5fdc84e6ebc311fe08785cf21, State: Initialized, Role: FOLLOWER
I20260811 12:46:28.541493 10404 consensus_queue.cc:260] T 99b21241f7304a26a7c07006ca24bdcd P aef6c0a5fdc84e6ebc311fe08785cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } }
I20260811 12:46:28.542105 10404 ts_tablet_manager.cc:1428] T 99b21241f7304a26a7c07006ca24bdcd P aef6c0a5fdc84e6ebc311fe08785cf21: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 12:46:28.543195 10437 ts_tablet_manager.cc:1918] T bdfbacd29fd549778d21c73f118cecbf P aef6c0a5fdc84e6ebc311fe08785cf21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:46:28.543485 10437 log.cc:1196] T bdfbacd29fd549778d21c73f118cecbf P aef6c0a5fdc84e6ebc311fe08785cf21: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/bdfbacd29fd549778d21c73f118cecbf
I20260811 12:46:28.544232 10437 ts_tablet_manager.cc:1939] T bdfbacd29fd549778d21c73f118cecbf P aef6c0a5fdc84e6ebc311fe08785cf21: Deleting consensus metadata
I20260811 12:46:28.545902 9505 catalog_manager.cc:4841] TS aef6c0a5fdc84e6ebc311fe08785cf21 (127.7.111.65:43759): tablet bdfbacd29fd549778d21c73f118cecbf (table test-table [id=64e67b0ec3434f3e9a52a3eb1c3d42e7]) successfully deleted
I20260811 12:46:28.821576 10415 raft_consensus.cc:482] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:46:28.821846 10415 raft_consensus.cc:504] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } }
I20260811 12:46:28.822556 10415 leader_election.cc:288] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aef6c0a5fdc84e6ebc311fe08785cf21 (127.7.111.65:43759), 5b61ba89b1524affa03175e80409de8d (127.7.111.66:37467)
I20260811 12:46:28.823184 9821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99b21241f7304a26a7c07006ca24bdcd" candidate_uuid: "47b60b3751854c1d87af5a75cae1bb2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" is_pre_election: true
I20260811 12:46:28.823299 10040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99b21241f7304a26a7c07006ca24bdcd" candidate_uuid: "47b60b3751854c1d87af5a75cae1bb2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b61ba89b1524affa03175e80409de8d" is_pre_election: true
I20260811 12:46:28.823486 9821 raft_consensus.cc:2454] T 99b21241f7304a26a7c07006ca24bdcd P aef6c0a5fdc84e6ebc311fe08785cf21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47b60b3751854c1d87af5a75cae1bb2b in term 0.
I20260811 12:46:28.823540 10040 raft_consensus.cc:2454] T 99b21241f7304a26a7c07006ca24bdcd P 5b61ba89b1524affa03175e80409de8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47b60b3751854c1d87af5a75cae1bb2b in term 0.
I20260811 12:46:28.823998 10159 leader_election.cc:302] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 47b60b3751854c1d87af5a75cae1bb2b, 5b61ba89b1524affa03175e80409de8d, aef6c0a5fdc84e6ebc311fe08785cf21; no voters:
I20260811 12:46:28.824360 10415 raft_consensus.cc:2781] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:46:28.824505 10415 raft_consensus.cc:482] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:46:28.824640 10415 raft_consensus.cc:3037] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:28.827132 10415 raft_consensus.cc:504] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } }
I20260811 12:46:28.827837 10415 leader_election.cc:288] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b [CANDIDATE]: Term 1 election: Requested vote from peers aef6c0a5fdc84e6ebc311fe08785cf21 (127.7.111.65:43759), 5b61ba89b1524affa03175e80409de8d (127.7.111.66:37467)
I20260811 12:46:28.828176 9821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99b21241f7304a26a7c07006ca24bdcd" candidate_uuid: "47b60b3751854c1d87af5a75cae1bb2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21"
I20260811 12:46:28.828176 10040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99b21241f7304a26a7c07006ca24bdcd" candidate_uuid: "47b60b3751854c1d87af5a75cae1bb2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b61ba89b1524affa03175e80409de8d"
I20260811 12:46:28.828384 10040 raft_consensus.cc:3037] T 99b21241f7304a26a7c07006ca24bdcd P 5b61ba89b1524affa03175e80409de8d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:28.828384 9821 raft_consensus.cc:3037] T 99b21241f7304a26a7c07006ca24bdcd P aef6c0a5fdc84e6ebc311fe08785cf21 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:28.831372 10040 raft_consensus.cc:2454] T 99b21241f7304a26a7c07006ca24bdcd P 5b61ba89b1524affa03175e80409de8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47b60b3751854c1d87af5a75cae1bb2b in term 1.
I20260811 12:46:28.831370 9821 raft_consensus.cc:2454] T 99b21241f7304a26a7c07006ca24bdcd P aef6c0a5fdc84e6ebc311fe08785cf21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47b60b3751854c1d87af5a75cae1bb2b in term 1.
I20260811 12:46:28.831827 10168 leader_election.cc:302] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47b60b3751854c1d87af5a75cae1bb2b, 5b61ba89b1524affa03175e80409de8d; no voters:
I20260811 12:46:28.832171 10415 raft_consensus.cc:2781] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:46:28.832345 10415 raft_consensus.cc:686] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b [term 1 LEADER]: Becoming Leader. State: Replica: 47b60b3751854c1d87af5a75cae1bb2b, State: Running, Role: LEADER
I20260811 12:46:28.832634 10415 consensus_queue.cc:237] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } }
I20260811 12:46:28.835493 9532 catalog_manager.cc:5495] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b reported cstate change: term changed from 0 to 1, leader changed from <none> to 47b60b3751854c1d87af5a75cae1bb2b (127.7.111.67). New cstate: current_term: 1 leader_uuid: "47b60b3751854c1d87af5a75cae1bb2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } health_report { overall_health: HEALTHY } } }
Aug 11 12:46:28 dist-test-slave-f0tn krb5kdc[9455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452386, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:28.912528 10455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:28.895688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:37631 (local address 127.0.0.1:46488)
0811 12:46:28.896410 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:28.896442 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:28.896598 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:28.896974 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:28.896978 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:28.897772 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:28.898068 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:28.898078 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.899159 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.899163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:28.900105 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:28.900117 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.900814 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.900821 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:28.900991 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:28.904431 (+ 3440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:28.904449 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:28.906812 (+ 2363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:28.910020 (+ 3208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:28.910032 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:28.910041 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:28.910441 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:28.910753 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:28.910758 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:28.910762 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:28.910896 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:28.911387 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:28.911395 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:28.911566 (+ 171us) client_negotiation.cc:769] Sending connection context
0811 12:46:28.911716 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 12:46:28.912041 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":399,"threads_started":1}
I20260811 12:46:28.912549 10456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:28.895734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46488 (local address 127.7.111.126:37631)
0811 12:46:28.896726 (+ 992us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:28.896729 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:28.896746 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:46:28.896784 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:28.896787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:28.896838 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:28.896938 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:28.898223 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.899024 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.900233 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:28.900637 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:28.900749 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:28.906984 (+ 6235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:28.906989 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:28.906991 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:28.907023 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:28.909920 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:28.910543 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:28.910549 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:28.910552 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:28.910662 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:28.910990 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:28.910995 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:28.910998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:28.911268 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:28.911393 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:28.911840 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 12:46:28.912176 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":91,"server-negotiator.queue_time_us":867,"thread_start_us":724,"threads_started":1}
I20260811 12:46:28.918807 9532 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:46488:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 12:46:28.920068 9532 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:46:28.927135 9778 tablet_service.cc:1467] Processing CreateTablet for tablet d82cb122d3654daab3780ae2576659b1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a107eb73b82f444a9b18433b3302aa8b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:46:28.927474 10007 tablet_service.cc:1467] Processing CreateTablet for tablet d82cb122d3654daab3780ae2576659b1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a107eb73b82f444a9b18433b3302aa8b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:46:28.927695 9778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d82cb122d3654daab3780ae2576659b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:28.927620 10247 tablet_service.cc:1467] Processing CreateTablet for tablet d82cb122d3654daab3780ae2576659b1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a107eb73b82f444a9b18433b3302aa8b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:46:28.928004 10007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d82cb122d3654daab3780ae2576659b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:28.928098 10247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d82cb122d3654daab3780ae2576659b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:28.935642 10403 tablet_bootstrap.cc:492] T d82cb122d3654daab3780ae2576659b1 P 47b60b3751854c1d87af5a75cae1bb2b: Bootstrap starting.
I20260811 12:46:28.935658 10404 tablet_bootstrap.cc:492] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21: Bootstrap starting.
I20260811 12:46:28.935963 10407 tablet_bootstrap.cc:492] T d82cb122d3654daab3780ae2576659b1 P 5b61ba89b1524affa03175e80409de8d: Bootstrap starting.
I20260811 12:46:28.937482 10404 tablet_bootstrap.cc:654] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:28.937597 10403 tablet_bootstrap.cc:654] T d82cb122d3654daab3780ae2576659b1 P 47b60b3751854c1d87af5a75cae1bb2b: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:28.937841 10407 tablet_bootstrap.cc:654] T d82cb122d3654daab3780ae2576659b1 P 5b61ba89b1524affa03175e80409de8d: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:28.947292 10404 tablet_bootstrap.cc:492] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21: No bootstrap required, opened a new log
I20260811 12:46:28.947461 10404 ts_tablet_manager.cc:1397] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.006s
I20260811 12:46:28.947507 10407 tablet_bootstrap.cc:492] T d82cb122d3654daab3780ae2576659b1 P 5b61ba89b1524affa03175e80409de8d: No bootstrap required, opened a new log
I20260811 12:46:28.947649 10407 ts_tablet_manager.cc:1397] T d82cb122d3654daab3780ae2576659b1 P 5b61ba89b1524affa03175e80409de8d: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260811 12:46:28.948256 10403 tablet_bootstrap.cc:492] T d82cb122d3654daab3780ae2576659b1 P 47b60b3751854c1d87af5a75cae1bb2b: No bootstrap required, opened a new log
I20260811 12:46:28.948216 10404 raft_consensus.cc:348] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } }
I20260811 12:46:28.948313 10407 raft_consensus.cc:348] T d82cb122d3654daab3780ae2576659b1 P 5b61ba89b1524affa03175e80409de8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } }
I20260811 12:46:28.948454 10403 ts_tablet_manager.cc:1397] T d82cb122d3654daab3780ae2576659b1 P 47b60b3751854c1d87af5a75cae1bb2b: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.009s
I20260811 12:46:28.948467 10404 raft_consensus.cc:374] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:28.948525 10407 raft_consensus.cc:374] T d82cb122d3654daab3780ae2576659b1 P 5b61ba89b1524affa03175e80409de8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:28.948540 10404 raft_consensus.cc:729] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aef6c0a5fdc84e6ebc311fe08785cf21, State: Initialized, Role: FOLLOWER
I20260811 12:46:28.948606 10407 raft_consensus.cc:729] T d82cb122d3654daab3780ae2576659b1 P 5b61ba89b1524affa03175e80409de8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b61ba89b1524affa03175e80409de8d, State: Initialized, Role: FOLLOWER
I20260811 12:46:28.948820 10404 consensus_queue.cc:260] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } }
I20260811 12:46:28.948837 10407 consensus_queue.cc:260] T d82cb122d3654daab3780ae2576659b1 P 5b61ba89b1524affa03175e80409de8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } }
I20260811 12:46:28.949189 10403 raft_consensus.cc:348] T d82cb122d3654daab3780ae2576659b1 P 47b60b3751854c1d87af5a75cae1bb2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } }
I20260811 12:46:28.949469 10403 raft_consensus.cc:374] T d82cb122d3654daab3780ae2576659b1 P 47b60b3751854c1d87af5a75cae1bb2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:28.949308 10411 tablet_replica.cc:439] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } } }
I20260811 12:46:28.949590 10403 raft_consensus.cc:729] T d82cb122d3654daab3780ae2576659b1 P 47b60b3751854c1d87af5a75cae1bb2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47b60b3751854c1d87af5a75cae1bb2b, State: Initialized, Role: FOLLOWER
I20260811 12:46:28.949622 10411 tablet_replica.cc:442] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:46:28.949992 10403 consensus_queue.cc:260] T d82cb122d3654daab3780ae2576659b1 P 47b60b3751854c1d87af5a75cae1bb2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } }
I20260811 12:46:28.950563 10403 ts_tablet_manager.cc:1428] T d82cb122d3654daab3780ae2576659b1 P 47b60b3751854c1d87af5a75cae1bb2b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 12:46:28.950512 10415 tablet_replica.cc:439] T d82cb122d3654daab3780ae2576659b1 P 47b60b3751854c1d87af5a75cae1bb2b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } } }
I20260811 12:46:28.950763 10415 tablet_replica.cc:442] T d82cb122d3654daab3780ae2576659b1 P 47b60b3751854c1d87af5a75cae1bb2b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:46:28.951087 10404 ts_tablet_manager.cc:1428] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260811 12:46:28.951417 10412 tablet_replica.cc:439] T d82cb122d3654daab3780ae2576659b1 P 5b61ba89b1524affa03175e80409de8d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } } }
I20260811 12:46:28.951722 10412 tablet_replica.cc:442] T d82cb122d3654daab3780ae2576659b1 P 5b61ba89b1524affa03175e80409de8d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:46:28.952533 10407 ts_tablet_manager.cc:1428] T d82cb122d3654daab3780ae2576659b1 P 5b61ba89b1524affa03175e80409de8d: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260811 12:46:29.089365 10411 raft_consensus.cc:482] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:46:29.089593 10411 raft_consensus.cc:504] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } }
I20260811 12:46:29.091270 10411 leader_election.cc:288] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b61ba89b1524affa03175e80409de8d (127.7.111.66:37467), 47b60b3751854c1d87af5a75cae1bb2b (127.7.111.67:45345)
I20260811 12:46:29.108436 10461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:29.091498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:37467 (local address 127.7.111.65:50917)
0811 12:46:29.091966 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:29.091985 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:29.092102 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:29.102500 (+ 10398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:29.102506 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:29.102556 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:29.102981 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:29.102993 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.104562 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.104568 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:29.106422 (+ 1854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:29.106434 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.107559 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.107566 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:29.107759 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:29.107793 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 12:46:29.107881 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 12:46:29.108014 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":10,"spinlock_wait_cycles":896,"thread_start_us":234,"threads_started":1}
I20260811 12:46:29.108453 10462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:29.091633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:50917 (local address 127.7.111.66:37467)
0811 12:46:29.102163 (+ 10530us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:29.102169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:29.102198 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:46:29.102269 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:29.102274 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:29.102342 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:29.103082 (+ 740us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:29.103263 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.104413 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.106570 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.107395 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.107553 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:29.107629 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:29.107938 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 12:46:29.108088 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10386,"thread_start_us":10288,"threads_started":1}
I20260811 12:46:29.108505 10463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:29.092108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:45345 (local address 127.7.111.65:46711)
0811 12:46:29.101676 (+ 9568us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:29.101699 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:29.101865 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:29.102340 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:29.102346 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:29.102375 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:29.102888 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:29.102906 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.104003 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.104009 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:29.105965 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:29.105981 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.107780 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.107786 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:29.107935 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:29.107964 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 12:46:29.108035 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 12:46:29.108146 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9430,"mutex_wait_us":34,"spinlock_wait_cycles":21632,"thread_start_us":9271,"threads_started":1}
I20260811 12:46:29.109056 10464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:29.092456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:46711 (local address 127.7.111.67:45345)
0811 12:46:29.101936 (+ 9480us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:29.101943 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:29.101964 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:46:29.102096 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:29.102101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:29.102176 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:29.102278 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:29.103097 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.103862 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.106153 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.107444 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.107612 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:29.107684 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:29.108608 (+ 924us) server_negotiation.cc:299] Negotiation successful
0811 12:46:29.108759 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9351,"thread_start_us":9236,"threads_started":1}
I20260811 12:46:29.109658 10040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82cb122d3654daab3780ae2576659b1" candidate_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b61ba89b1524affa03175e80409de8d" is_pre_election: true
I20260811 12:46:29.109956 10040 raft_consensus.cc:2454] T d82cb122d3654daab3780ae2576659b1 P 5b61ba89b1524affa03175e80409de8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aef6c0a5fdc84e6ebc311fe08785cf21 in term 0.
I20260811 12:46:29.109962 10287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82cb122d3654daab3780ae2576659b1" candidate_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47b60b3751854c1d87af5a75cae1bb2b" is_pre_election: true
I20260811 12:46:29.110422 10287 raft_consensus.cc:2454] T d82cb122d3654daab3780ae2576659b1 P 47b60b3751854c1d87af5a75cae1bb2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aef6c0a5fdc84e6ebc311fe08785cf21 in term 0.
I20260811 12:46:29.110476 9693 leader_election.cc:302] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b61ba89b1524affa03175e80409de8d, aef6c0a5fdc84e6ebc311fe08785cf21; no voters:
I20260811 12:46:29.110901 10411 raft_consensus.cc:2781] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:46:29.111037 10411 raft_consensus.cc:482] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:46:29.111174 10411 raft_consensus.cc:3037] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:29.114439 10411 raft_consensus.cc:504] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } }
I20260811 12:46:29.114969 10411 leader_election.cc:288] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21 [CANDIDATE]: Term 1 election: Requested vote from peers 5b61ba89b1524affa03175e80409de8d (127.7.111.66:37467), 47b60b3751854c1d87af5a75cae1bb2b (127.7.111.67:45345)
I20260811 12:46:29.115396 10040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82cb122d3654daab3780ae2576659b1" candidate_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b61ba89b1524affa03175e80409de8d"
I20260811 12:46:29.115483 10287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82cb122d3654daab3780ae2576659b1" candidate_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47b60b3751854c1d87af5a75cae1bb2b"
I20260811 12:46:29.115643 10040 raft_consensus.cc:3037] T d82cb122d3654daab3780ae2576659b1 P 5b61ba89b1524affa03175e80409de8d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:29.115751 10287 raft_consensus.cc:3037] T d82cb122d3654daab3780ae2576659b1 P 47b60b3751854c1d87af5a75cae1bb2b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:29.118639 10040 raft_consensus.cc:2454] T d82cb122d3654daab3780ae2576659b1 P 5b61ba89b1524affa03175e80409de8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aef6c0a5fdc84e6ebc311fe08785cf21 in term 1.
I20260811 12:46:29.118727 10287 raft_consensus.cc:2454] T d82cb122d3654daab3780ae2576659b1 P 47b60b3751854c1d87af5a75cae1bb2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aef6c0a5fdc84e6ebc311fe08785cf21 in term 1.
I20260811 12:46:29.119069 9693 leader_election.cc:302] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b61ba89b1524affa03175e80409de8d, aef6c0a5fdc84e6ebc311fe08785cf21; no voters:
I20260811 12:46:29.119328 10411 raft_consensus.cc:2781] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:46:29.119755 10411 raft_consensus.cc:686] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21 [term 1 LEADER]: Becoming Leader. State: Replica: aef6c0a5fdc84e6ebc311fe08785cf21, State: Running, Role: LEADER
I20260811 12:46:29.120095 10411 consensus_queue.cc:237] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } }
I20260811 12:46:29.122027 10465 tablet_replica.cc:439] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21: TxnStatusTablet state changed. Reason: New leader aef6c0a5fdc84e6ebc311fe08785cf21. Latest consensus state: current_term: 1 leader_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } } }
I20260811 12:46:29.122629 10465 tablet_replica.cc:442] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:46:29.123692 9534 catalog_manager.cc:5495] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21 reported cstate change: term changed from 0 to 1, leader changed from <none> to aef6c0a5fdc84e6ebc311fe08785cf21 (127.7.111.65). New cstate: current_term: 1 leader_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } health_report { overall_health: UNKNOWN } } }
I20260811 12:46:29.124177 10468 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 12:46:29.124469 10468 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 12:46:29.144285 10455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:29.135080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:43759 (local address 127.0.0.1:34808)
0811 12:46:29.135328 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:29.135349 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:29.135490 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:29.138364 (+ 2874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:29.138368 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:29.138379 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:46:29.138544 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:29.138551 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.140235 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.140238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:29.141227 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:29.141238 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.142348 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.142353 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:29.142549 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:29.142612 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:46:29.143828 (+ 1216us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:46:29.143848 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 12:46:29.143902 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 12:46:29.144031 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 12:46:29.144490 10473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:29.135221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34808 (local address 127.7.111.65:43759)
0811 12:46:29.137848 (+ 2627us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:29.137854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:29.137881 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:46:29.137989 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:29.137995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:29.138113 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:29.138336 (+ 223us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:46:29.138737 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.140112 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.141373 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.142165 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.142336 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:29.142900 (+ 564us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:46:29.143724 (+ 824us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:46:29.143881 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:29.143933 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 12:46:29.144019 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2478,"thread_start_us":2327,"threads_started":1}
I20260811 12:46:29.149283 10040 raft_consensus.cc:1261] T d82cb122d3654daab3780ae2576659b1 P 5b61ba89b1524affa03175e80409de8d [term 1 FOLLOWER]: Refusing update from remote peer aef6c0a5fdc84e6ebc311fe08785cf21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:46:29.149816 10287 raft_consensus.cc:1261] T d82cb122d3654daab3780ae2576659b1 P 47b60b3751854c1d87af5a75cae1bb2b [term 1 FOLLOWER]: Refusing update from remote peer aef6c0a5fdc84e6ebc311fe08785cf21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:46:29.150076 10465 consensus_queue.cc:1035] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:46:29.150617 10411 consensus_queue.cc:1035] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:46:29.155048 10412 tablet_replica.cc:439] T d82cb122d3654daab3780ae2576659b1 P 5b61ba89b1524affa03175e80409de8d: TxnStatusTablet state changed. Reason: New leader aef6c0a5fdc84e6ebc311fe08785cf21. Latest consensus state: current_term: 1 leader_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } } }
I20260811 12:46:29.155403 10412 tablet_replica.cc:442] T d82cb122d3654daab3780ae2576659b1 P 5b61ba89b1524affa03175e80409de8d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:46:29.158387 10480 mvcc.cc:205] Tried to move back new op lower bound from 7317308985948549120 to 7317308985839472640. Current Snapshot: MvccSnapshot[applied={T|T < 7317308985948549120}]
I20260811 12:46:29.161958 10465 tablet_replica.cc:439] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } } }
I20260811 12:46:29.162364 10465 tablet_replica.cc:442] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:46:29.175204 10412 tablet_replica.cc:439] T d82cb122d3654daab3780ae2576659b1 P 5b61ba89b1524affa03175e80409de8d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } } }
I20260811 12:46:29.175511 10412 tablet_replica.cc:442] T d82cb122d3654daab3780ae2576659b1 P 5b61ba89b1524affa03175e80409de8d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:46:29.184365 10415 tablet_replica.cc:439] T d82cb122d3654daab3780ae2576659b1 P 47b60b3751854c1d87af5a75cae1bb2b: TxnStatusTablet state changed. Reason: New leader aef6c0a5fdc84e6ebc311fe08785cf21. Latest consensus state: current_term: 1 leader_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } } }
I20260811 12:46:29.184687 10415 tablet_replica.cc:442] T d82cb122d3654daab3780ae2576659b1 P 47b60b3751854c1d87af5a75cae1bb2b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:46:29.186470 10411 tablet_replica.cc:439] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } } }
I20260811 12:46:29.186795 10411 tablet_replica.cc:442] T d82cb122d3654daab3780ae2576659b1 P aef6c0a5fdc84e6ebc311fe08785cf21: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:46:29.194835 10415 tablet_replica.cc:439] T d82cb122d3654daab3780ae2576659b1 P 47b60b3751854c1d87af5a75cae1bb2b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 } } peers { permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 } } peers { permanent_uuid: "47b60b3751854c1d87af5a75cae1bb2b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45345 } } }
I20260811 12:46:29.195297 10415 tablet_replica.cc:442] T d82cb122d3654daab3780ae2576659b1 P 47b60b3751854c1d87af5a75cae1bb2b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:46:29.203004 10473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:29.196084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:37813 (local address 127.7.111.65:43759)
0811 12:46:29.196264 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:29.196271 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:29.196983 (+ 712us) server_negotiation.cc:407] Connection header received
0811 12:46:29.197174 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:29.197178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:29.197251 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:29.197319 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:46:29.197894 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.198898 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.201054 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.201735 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.201840 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:29.202378 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:46:29.202480 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:46:29.202558 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:29.202709 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 12:46:29.202771 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 12:46:29.203280 10495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:29.195944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:43759 (local address 127.7.111.67:37813)
0811 12:46:29.196871 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:29.196893 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:29.197052 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:29.197360 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:29.197365 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:29.197400 (+ 35us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:46:29.197741 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:29.197756 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.199043 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.199049 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:29.200896 (+ 1847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:29.200913 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.201917 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.201923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:29.202103 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:29.202194 (+ 91us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:46:29.202576 (+ 382us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:46:29.202589 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 12:46:29.202668 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 12:46:29.202808 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":550,"threads_started":1}
I20260811 12:46:29.214617 10040 raft_consensus.cc:1261] T 99b21241f7304a26a7c07006ca24bdcd P 5b61ba89b1524affa03175e80409de8d [term 1 FOLLOWER]: Refusing update from remote peer 47b60b3751854c1d87af5a75cae1bb2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:46:29.214707 9821 raft_consensus.cc:1261] T 99b21241f7304a26a7c07006ca24bdcd P aef6c0a5fdc84e6ebc311fe08785cf21 [term 1 FOLLOWER]: Refusing update from remote peer 47b60b3751854c1d87af5a75cae1bb2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:46:29.215189 10415 consensus_queue.cc:1035] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b61ba89b1524affa03175e80409de8d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:46:29.215471 10410 consensus_queue.cc:1035] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "aef6c0a5fdc84e6ebc311fe08785cf21" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:46:29.221405 10481 mvcc.cc:205] Tried to move back new op lower bound from 7317308986217426944 to 7317308984660230144. Current Snapshot: MvccSnapshot[applied={T|T < 7317308986217426944}]
I20260811 12:46:29.256240 10464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:29.245637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:39077 (local address 127.7.111.67:45345)
0811 12:46:29.245840 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:29.245847 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:29.250204 (+ 4357us) server_negotiation.cc:407] Connection header received
0811 12:46:29.250546 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:29.250551 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:29.250654 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:29.250739 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:46:29.251464 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.252265 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.254047 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.254654 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.254806 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:29.255395 (+ 589us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:46:29.255576 (+ 181us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:46:29.255651 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:29.255906 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 12:46:29.255988 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:46:29.256608 10512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:29.245549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:45345 (local address 127.7.111.65:39077)
0811 12:46:29.250083 (+ 4534us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:29.250107 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:29.250284 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:29.250769 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:29.250775 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:29.250787 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:46:29.251266 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:29.251282 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.252413 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.252417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:29.253875 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:29.253890 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.254861 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.254870 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:29.255085 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:29.255165 (+ 80us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:46:29.255770 (+ 605us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:46:29.255783 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 12:46:29.255887 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 12:46:29.256034 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4292,"thread_start_us":4074,"threads_started":1}
I20260811 12:46:29.288571 9532 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:46:29.288810 9532 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:46:29.291985 9532 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60cc079943e74141ac1e649304bdeeee: Sending DeleteTablet for 3 replicas of tablet 99b21241f7304a26a7c07006ca24bdcd
I20260811 12:46:29.292834 9778 tablet_service.cc:1514] Processing DeleteTablet for tablet 99b21241f7304a26a7c07006ca24bdcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:29 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:34804
I20260811 12:46:29.292876 10247 tablet_service.cc:1514] Processing DeleteTablet for tablet 99b21241f7304a26a7c07006ca24bdcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:29 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:41314
I20260811 12:46:29.293601 10007 tablet_service.cc:1514] Processing DeleteTablet for tablet 99b21241f7304a26a7c07006ca24bdcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:29 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:42424
I20260811 12:46:29.300745 10516 tablet_replica.cc:330] T 99b21241f7304a26a7c07006ca24bdcd P aef6c0a5fdc84e6ebc311fe08785cf21: stopping tablet replica
I20260811 12:46:29.300995 10516 raft_consensus.cc:2229] T 99b21241f7304a26a7c07006ca24bdcd P aef6c0a5fdc84e6ebc311fe08785cf21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:46:29.301880 10520 tablet_replica.cc:330] T 99b21241f7304a26a7c07006ca24bdcd P 5b61ba89b1524affa03175e80409de8d: stopping tablet replica
I20260811 12:46:29.302273 10520 raft_consensus.cc:2229] T 99b21241f7304a26a7c07006ca24bdcd P 5b61ba89b1524affa03175e80409de8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:46:29.302601 10516 raft_consensus.cc:2258] T 99b21241f7304a26a7c07006ca24bdcd P aef6c0a5fdc84e6ebc311fe08785cf21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:46:29.302850 10520 raft_consensus.cc:2258] T 99b21241f7304a26a7c07006ca24bdcd P 5b61ba89b1524affa03175e80409de8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:46:29.304222 10520 ts_tablet_manager.cc:1905] T 99b21241f7304a26a7c07006ca24bdcd P 5b61ba89b1524affa03175e80409de8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:46:29.307556 10516 ts_tablet_manager.cc:1905] T 99b21241f7304a26a7c07006ca24bdcd P aef6c0a5fdc84e6ebc311fe08785cf21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:46:29.309940 10514 tablet_replica.cc:330] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b: stopping tablet replica
I20260811 12:46:29.310190 10514 raft_consensus.cc:2229] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:46:29.310727 10514 raft_consensus.cc:2258] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:46:29.312903 10520 ts_tablet_manager.cc:1918] T 99b21241f7304a26a7c07006ca24bdcd P 5b61ba89b1524affa03175e80409de8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:46:29.313119 10520 log.cc:1196] T 99b21241f7304a26a7c07006ca24bdcd P 5b61ba89b1524affa03175e80409de8d: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/99b21241f7304a26a7c07006ca24bdcd
I20260811 12:46:29.313941 10520 ts_tablet_manager.cc:1939] T 99b21241f7304a26a7c07006ca24bdcd P 5b61ba89b1524affa03175e80409de8d: Deleting consensus metadata
I20260811 12:46:29.315225 9506 catalog_manager.cc:4841] TS 5b61ba89b1524affa03175e80409de8d (127.7.111.66:37467): tablet 99b21241f7304a26a7c07006ca24bdcd (table test-table [id=5dcc1a39e432422c939671dacbad95f1]) successfully deleted
I20260811 12:46:29.317485 10516 ts_tablet_manager.cc:1918] T 99b21241f7304a26a7c07006ca24bdcd P aef6c0a5fdc84e6ebc311fe08785cf21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:46:29.317664 10516 log.cc:1196] T 99b21241f7304a26a7c07006ca24bdcd P aef6c0a5fdc84e6ebc311fe08785cf21: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/99b21241f7304a26a7c07006ca24bdcd
I20260811 12:46:29.318166 10516 ts_tablet_manager.cc:1939] T 99b21241f7304a26a7c07006ca24bdcd P aef6c0a5fdc84e6ebc311fe08785cf21: Deleting consensus metadata
I20260811 12:46:29.319325 10514 ts_tablet_manager.cc:1905] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:46:29.320032 9505 catalog_manager.cc:4841] TS aef6c0a5fdc84e6ebc311fe08785cf21 (127.7.111.65:43759): tablet 99b21241f7304a26a7c07006ca24bdcd (table test-table [id=5dcc1a39e432422c939671dacbad95f1]) successfully deleted
Aug 11 12:46:29 dist-test-slave-f0tn krb5kdc[9455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452388, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.111.65@KRBTEST.COM
I20260811 12:46:29.325492 10514 ts_tablet_manager.cc:1918] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:46:29.325626 10514 log.cc:1196] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/99b21241f7304a26a7c07006ca24bdcd
I20260811 12:46:29.326068 10514 ts_tablet_manager.cc:1939] T 99b21241f7304a26a7c07006ca24bdcd P 47b60b3751854c1d87af5a75cae1bb2b: Deleting consensus metadata
I20260811 12:46:29.327099 9506 catalog_manager.cc:4841] TS 47b60b3751854c1d87af5a75cae1bb2b (127.7.111.67:45345): tablet 99b21241f7304a26a7c07006ca24bdcd (table test-table [id=5dcc1a39e432422c939671dacbad95f1]) successfully deleted
I20260811 12:46:29.334447 10473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:29.304660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34810 (local address 127.7.111.65:43759)
0811 12:46:29.304855 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:29.304861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:29.312299 (+ 7438us) server_negotiation.cc:407] Connection header received
0811 12:46:29.312600 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:29.312606 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:29.312691 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:29.312776 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:29.315220 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.316424 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.318532 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.319336 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.319485 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:29.326713 (+ 7228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:29.326731 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:29.326753 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:29.326807 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:29.331075 (+ 4268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:29.331809 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:29.331821 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:29.331824 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:29.332008 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:29.332363 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:29.332369 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:29.332374 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:29.332922 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:29.333425 (+ 503us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:29.333641 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 12:46:29.333873 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 12:46:29.335335 9897 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34810
I20260811 12:46:29.352876 10456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:29.338639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46494 (local address 127.7.111.126:37631)
0811 12:46:29.338939 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:29.338945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:29.339072 (+ 127us) server_negotiation.cc:407] Connection header received
0811 12:46:29.339216 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:29.339222 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:29.339310 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:29.339426 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:29.340663 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.341907 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.343163 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.343953 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.344103 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:29.346428 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:29.346436 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:29.346441 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:29.346489 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:29.350196 (+ 3707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:29.350805 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:29.350811 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:29.350815 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:29.350974 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:29.351290 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:29.351295 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:29.351299 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:29.351734 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:29.352015 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:29.352222 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 12:46:29.352470 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
W20260811 12:46:29.353569 9532 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46494
I20260811 12:46:29.356192 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 9667
I20260811 12:46:29.370293 10473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:29.357114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34822 (local address 127.7.111.65:43759)
0811 12:46:29.357330 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:29.357342 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:29.357589 (+ 247us) server_negotiation.cc:407] Connection header received
0811 12:46:29.357714 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:29.357720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:29.357817 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:29.357915 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:29.358948 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.360035 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.361199 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.362082 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.362240 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:29.364566 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:29.364572 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:29.364575 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:29.364611 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:29.367512 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:29.368157 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:29.368163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:29.368165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:29.368299 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:29.368639 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:29.368642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:29.368644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:29.368973 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:29.369304 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:29.369647 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 12:46:29.369904 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 12:46:29.370970 9897 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34822
W20260811 12:46:29.371619 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 9667: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:46:29.371776 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 9667
I20260811 12:46:29.407227 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 9909
Aug 11 12:46:29 dist-test-slave-f0tn krb5kdc[9455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452388, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.111.66@KRBTEST.COM
I20260811 12:46:29.421415 10462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:29.408032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42438 (local address 127.7.111.66:37467)
0811 12:46:29.408251 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:29.408256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:29.408275 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:46:29.408358 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:29.408364 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:29.408433 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:29.408520 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:29.409751 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.410594 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.411767 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.412150 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.412244 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:29.416338 (+ 4094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:29.416353 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:29.416370 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:29.416404 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:29.419133 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:29.419699 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:29.419706 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:29.419708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:29.419834 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:29.420099 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:29.420102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:29.420104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:29.420470 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:29.420719 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:29.420872 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 12:46:29.421059 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 12:46:29.422159 10114 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42438
W20260811 12:46:29.422775 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 9909: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:46:29.422924 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 9909
I20260811 12:46:29.451822 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 10131
Aug 11 12:46:29 dist-test-slave-f0tn krb5kdc[9455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452388, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.111.67@KRBTEST.COM
I20260811 12:46:29.465893 10464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:29.452545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41326 (local address 127.7.111.67:45345)
0811 12:46:29.452717 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:29.452721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:29.452740 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:46:29.452895 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:29.452899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:29.452963 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:29.453049 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:29.454357 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.455049 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.456279 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:29.456688 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:29.456779 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:29.460649 (+ 3870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:29.460666 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:29.460698 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:29.460736 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:29.463513 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:29.464065 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:29.464072 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:29.464074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:29.464225 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:29.464537 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:29.464543 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:29.464546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:29.464958 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:29.465185 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:29.465446 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 12:46:29.465624 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 12:46:29.466557 10365 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41326
W20260811 12:46:29.469496 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 10131: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:46:29.469645 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 10131
I20260811 12:46:29.509238 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 9480
I20260811 12:46:29.510375 9618 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:29.627182 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 9480
2026-08-11T12:46:29Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5002 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:46:29 dist-test-slave-f0tn krb5kdc[10538](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:46:29 dist-test-slave-f0tn krb5kdc[10538](info): set up 2 sockets
Aug 11 12:46:29 dist-test-slave-f0tn krb5kdc[10538](info): commencing operation
krb5kdc: starting...
W20260811 12:46:30.853147 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:46:30 dist-test-slave-f0tn krb5kdc[10538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:46:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:46:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:46:31.056102 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:32907
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:34719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:32907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:31.381848 10564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:31.382182 10564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:31.382323 10564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:31.389611 10564 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:46:31.389726 10564 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 12:46:31.389798 10564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:31.389890 10564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:31.389955 10564 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:46:31.390038 10564 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:46:31.400125 10564 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:34719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:32907
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:32907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:31.401489 10564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:31.402834 10564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:31.414388 10571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:31.414315 10572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:31.414347 10575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:31.415470 10564 server_base.cc:1034] running on GCE node
I20260811 12:46:31.416188 10564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:31.417534 10564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:31.418758 10564 hybrid_clock.cc:648] HybridClock initialized: now 1786452391418678 us; error 50 us; skew 500 ppm
I20260811 12:46:31.419062 10564 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:31 dist-test-slave-f0tn krb5kdc[10538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452391, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:31.422120 10564 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:46:31.423473 10564 webserver.cc:460] Webserver started at http://127.7.111.126:39183/ using document root <none> and password file <none>
I20260811 12:46:31.424137 10564 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:31.424235 10564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:31.424530 10564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:31.427459 10564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "0c918b7949a64e909d122c6801244706"
format_stamp: "Formatted at 2026-08-11 12:46:31 on dist-test-slave-f0tn"
server_key: "7ea41eab033868f73cdc25373347f982"
server_key_iv: "551d590fa2383f8d672828b3a276d21b"
server_key_version: "encryptionkey@0"
I20260811 12:46:31.428180 10564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "0c918b7949a64e909d122c6801244706"
format_stamp: "Formatted at 2026-08-11 12:46:31 on dist-test-slave-f0tn"
server_key: "7ea41eab033868f73cdc25373347f982"
server_key_iv: "551d590fa2383f8d672828b3a276d21b"
server_key_version: "encryptionkey@0"
I20260811 12:46:31.433030 10564 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 12:46:31.436293 10587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:31.437549 10564 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:46:31.437717 10564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "0c918b7949a64e909d122c6801244706"
format_stamp: "Formatted at 2026-08-11 12:46:31 on dist-test-slave-f0tn"
server_key: "7ea41eab033868f73cdc25373347f982"
server_key_iv: "551d590fa2383f8d672828b3a276d21b"
server_key_version: "encryptionkey@0"
I20260811 12:46:31.437904 10564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:31.463414 10564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:31.467044 10564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:31.467289 10564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:31.488066 10564 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:32907
I20260811 12:46:31.488095 10689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:32907 every 8 connection(s)
I20260811 12:46:31.489444 10564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:46:31.493513 10691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:31.495222 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 10564
I20260811 12:46:31.495460 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:46:31.496439 7613 external_mini_cluster.cc:1428] Setting key 548e3481291242dd16f60f1d196dd3a8
I20260811 12:46:31.509284 10691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706: Bootstrap starting.
Aug 11 12:46:31 dist-test-slave-f0tn krb5kdc[10538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:31.513649 10691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:31.514952 10691 log.cc:824] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706: Log is configured to *not* fsync() on all Append() calls
I20260811 12:46:31.518216 10691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706: No bootstrap required, opened a new log
I20260811 12:46:31.521903 10698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:31.498665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51326 (local address 127.7.111.126:32907)
0811 12:46:31.499516 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:31.499530 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:31.499576 (+ 46us) server_negotiation.cc:407] Connection header received
0811 12:46:31.501276 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:31.501306 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:31.501795 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:31.502144 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:31.504209 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:31.505624 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:31.507570 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:31.508691 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:31.508888 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:31.514402 (+ 5514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:31.514411 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:31.514428 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:31.514468 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:31.517531 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:31.518411 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:31.518424 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:31.518433 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:31.518636 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:31.519076 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:31.519079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:31.519081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:31.519613 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:31.519844 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:31.520347 (+ 503us) server_negotiation.cc:299] Negotiation successful
0811 12:46:31.520727 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":411,"threads_started":1}
I20260811 12:46:31.527551 10691 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c918b7949a64e909d122c6801244706" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 32907 } }
I20260811 12:46:31.527954 10691 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:31.528075 10691 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c918b7949a64e909d122c6801244706, State: Initialized, Role: FOLLOWER
I20260811 12:46:31.528703 10691 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c918b7949a64e909d122c6801244706" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 32907 } }
I20260811 12:46:31.528957 10691 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:46:31.529106 10691 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:46:31.529295 10691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:31.531984 10691 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c918b7949a64e909d122c6801244706" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 32907 } }
I20260811 12:46:31.532557 10691 leader_election.cc:302] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c918b7949a64e909d122c6801244706; no voters:
I20260811 12:46:31.533186 10691 leader_election.cc:288] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:46:31.533327 10702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:46:31.534035 10702 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706 [term 1 LEADER]: Becoming Leader. State: Replica: 0c918b7949a64e909d122c6801244706, State: Running, Role: LEADER
I20260811 12:46:31.534603 10702 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c918b7949a64e909d122c6801244706" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 32907 } }
I20260811 12:46:31.535555 10691 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:46:31.537724 10703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c918b7949a64e909d122c6801244706. Latest consensus state: current_term: 1 leader_uuid: "0c918b7949a64e909d122c6801244706" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c918b7949a64e909d122c6801244706" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 32907 } } }
I20260811 12:46:31.538130 10703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706 [sys.catalog]: This master's current role is: LEADER
I20260811 12:46:31.543105 10704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c918b7949a64e909d122c6801244706" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c918b7949a64e909d122c6801244706" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 32907 } } }
I20260811 12:46:31.543362 10704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706 [sys.catalog]: This master's current role is: LEADER
I20260811 12:46:31.544113 10712 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:46:31.549198 10712 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:46:31.558506 10712 catalog_manager.cc:1347] Generated new cluster ID: 8f20aa4b994243869d862c694456c104
I20260811 12:46:31.558627 10712 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:46:31.598902 10712 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:46:31.600564 10712 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:46:31.610267 10712 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706: Generated new TSK 0
I20260811 12:46:31.611073 10712 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.111.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
I20260811 12:46:31.672644 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:32907
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:34719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:32.008664 10736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:32.008952 10736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:32.009076 10736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:32.016606 10736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:32.016714 10736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:32.016808 10736 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 12:46:32.016858 10736 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:46:32.016927 10736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:46:32.026968 10736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:34719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.111.126:32907
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:32.028277 10736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:32.029667 10736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:32.042654 10744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:32.042745 10748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:32.042686 10746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:32.043673 10736 server_base.cc:1034] running on GCE node
I20260811 12:46:32.044273 10736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:32.045148 10736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:32.046339 10736 hybrid_clock.cc:648] HybridClock initialized: now 1786452392046273 us; error 54 us; skew 500 ppm
I20260811 12:46:32.046648 10736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:32 dist-test-slave-f0tn krb5kdc[10538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452392, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:32.049722 10736 init.cc:377] Logged in from keytab as kudu/127.7.111.65@KRBTEST.COM (short username kudu)
I20260811 12:46:32.051249 10736 webserver.cc:460] Webserver started at http://127.7.111.65:39105/ using document root <none> and password file <none>
I20260811 12:46:32.051872 10736 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:32.051971 10736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:32.052258 10736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:32.055202 10736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "6ff8914aa93c40f68d4a52bc8271e0c2"
format_stamp: "Formatted at 2026-08-11 12:46:32 on dist-test-slave-f0tn"
server_key: "fc65cba7ed9f5729a634ee2f29aa9c8c"
server_key_iv: "149d74638e5966bb7a4edba4189cceeb"
server_key_version: "encryptionkey@0"
I20260811 12:46:32.055930 10736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "6ff8914aa93c40f68d4a52bc8271e0c2"
format_stamp: "Formatted at 2026-08-11 12:46:32 on dist-test-slave-f0tn"
server_key: "fc65cba7ed9f5729a634ee2f29aa9c8c"
server_key_iv: "149d74638e5966bb7a4edba4189cceeb"
server_key_version: "encryptionkey@0"
I20260811 12:46:32.061163 10736 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 12:46:32.064574 10760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:32.065883 10736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:46:32.066039 10736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "6ff8914aa93c40f68d4a52bc8271e0c2"
format_stamp: "Formatted at 2026-08-11 12:46:32 on dist-test-slave-f0tn"
server_key: "fc65cba7ed9f5729a634ee2f29aa9c8c"
server_key_iv: "149d74638e5966bb7a4edba4189cceeb"
server_key_version: "encryptionkey@0"
I20260811 12:46:32.066229 10736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:32.082327 10736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:32.086655 10736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:32.086994 10736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:32.092592 10736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:32.092783 10736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 12:46:32.092962 10736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:32.093107 10736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:46:32 dist-test-slave-f0tn krb5kdc[10538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452392, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:32.127478 10774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:32.090755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:56081 (local address 127.7.111.126:32907)
0811 12:46:32.107897 (+ 17142us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:32.107902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:32.107922 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:46:32.108159 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:32.108165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:32.108272 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:32.108388 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:32.111297 (+ 2909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:32.112443 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:32.115193 (+ 2750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:32.115730 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:32.115835 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:32.120936 (+ 5101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:32.120943 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:32.120948 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:32.120992 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:32.124112 (+ 3120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:32.125013 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:32.125019 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:32.125023 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:32.125148 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:32.125729 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:32.125734 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:32.125738 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:32.126008 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:32.126148 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:32.126881 (+ 733us) server_negotiation.cc:299] Negotiation successful
0811 12:46:32.127106 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17031,"thread_start_us":17156,"threads_started":1}
I20260811 12:46:32.128770 10775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:32.091344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:32907 (local address 127.7.111.65:56081)
0811 12:46:32.103415 (+ 12071us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:32.103464 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:32.105259 (+ 1795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:32.108769 (+ 3510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:32.108777 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:32.110241 (+ 1464us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:32.111043 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:32.111076 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:32.112652 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:32.112657 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:32.114988 (+ 2331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:32.115009 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:32.116184 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:32.116191 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:32.116520 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:32.117302 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:32.117342 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:32.120644 (+ 3302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:32.124256 (+ 3612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:32.124269 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:32.124286 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:32.124885 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:32.125256 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:32.125261 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:32.125268 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:32.125603 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:32.126139 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:32.126146 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:32.126510 (+ 364us) client_negotiation.cc:769] Sending connection context
0811 12:46:32.126745 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 12:46:32.127287 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11780,"thread_start_us":11559,"threads_started":1}
I20260811 12:46:32.151697 10882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:32.137046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:32907 (local address 127.7.111.65:57425)
0811 12:46:32.137811 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:32.137839 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:32.137981 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:32.138300 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:32.138305 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:32.138905 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:32.139299 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:32.139313 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:32.140680 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:32.140702 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:32.142583 (+ 1881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:32.142599 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:32.143578 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:32.143585 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:32.143787 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:32.144405 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:32.144422 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:32.145196 (+ 774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:32.148599 (+ 3403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:32.148609 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:32.148613 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:32.149218 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:32.149662 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:32.149667 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:32.149679 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:32.149832 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:32.150512 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:32.150521 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:32.150704 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 12:46:32.150854 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 12:46:32.151207 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":437,"threads_started":1}
I20260811 12:46:32.152595 10774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:32.136898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:57425 (local address 127.7.111.126:32907)
0811 12:46:32.137083 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:32.137088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:32.137914 (+ 826us) server_negotiation.cc:407] Connection header received
0811 12:46:32.138098 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:32.138104 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:32.138181 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:32.138260 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:32.139450 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:32.140537 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:32.142760 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:32.143388 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:32.143504 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:32.145403 (+ 1899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:32.145410 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:32.145415 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:32.145454 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:32.148491 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:32.149330 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:32.149336 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:32.149340 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:32.149548 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:32.149965 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:32.149970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:32.149974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:32.150311 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:32.150448 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:32.152018 (+ 1570us) server_negotiation.cc:299] Negotiation successful
0811 12:46:32.152254 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:46:32.163429 10736 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:36087
I20260811 12:46:32.163501 10969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:36087 every 8 connection(s)
I20260811 12:46:32.165022 10736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:46:32.167220 10970 heartbeater.cc:347] Connected to a master server at 127.7.111.126:32907
I20260811 12:46:32.167444 10970 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:32.168087 10970 heartbeater.cc:511] Master 127.7.111.126:32907 requested a full tablet report, sending...
I20260811 12:46:32.168664 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 10736
I20260811 12:46:32.168864 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:46:32.169616 7613 external_mini_cluster.cc:1428] Setting key d64fe18dc7b57d038c1ec4050380b6a6
I20260811 12:46:32.170274 10615 ts_manager.cc:194] Registered new tserver with Master: 6ff8914aa93c40f68d4a52bc8271e0c2 (127.7.111.65:36087)
I20260811 12:46:32.171924 10615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:56081
WARNING: no policy specified for kudu/127.7.111.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
I20260811 12:46:32.216327 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:32907
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:34719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:32.540663 10977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:32.540951 10977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:32.541107 10977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:32.548314 10977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:32.548421 10977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:32.548542 10977 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 12:46:32.548602 10977 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:46:32.548705 10977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:46:32.558435 10977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:34719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.111.126:32907
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:32.559690 10977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:32.561021 10977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:32.574895 10984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:32.574975 10989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:32.574940 10985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:32.575736 10977 server_base.cc:1034] running on GCE node
I20260811 12:46:32.576354 10977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:32.577220 10977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:32.578423 10977 hybrid_clock.cc:648] HybridClock initialized: now 1786452392578365 us; error 50 us; skew 500 ppm
I20260811 12:46:32.578722 10977 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:32 dist-test-slave-f0tn krb5kdc[10538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452392, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:32.581756 10977 init.cc:377] Logged in from keytab as kudu/127.7.111.66@KRBTEST.COM (short username kudu)
I20260811 12:46:32.583179 10977 webserver.cc:460] Webserver started at http://127.7.111.66:37599/ using document root <none> and password file <none>
I20260811 12:46:32.583847 10977 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:32.583966 10977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:32.584282 10977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:32.587234 10977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "4955defa995f4f5f8c510f1ad24c1464"
format_stamp: "Formatted at 2026-08-11 12:46:32 on dist-test-slave-f0tn"
server_key: "b95df198e2d9d8db851c8dc6e2cd4ed9"
server_key_iv: "3052a20663602b3bb2ae9eef4bd46162"
server_key_version: "encryptionkey@0"
I20260811 12:46:32.587960 10977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "4955defa995f4f5f8c510f1ad24c1464"
format_stamp: "Formatted at 2026-08-11 12:46:32 on dist-test-slave-f0tn"
server_key: "b95df198e2d9d8db851c8dc6e2cd4ed9"
server_key_iv: "3052a20663602b3bb2ae9eef4bd46162"
server_key_version: "encryptionkey@0"
I20260811 12:46:32.593124 10977 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:46:32.596714 11001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:32.598055 10977 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:46:32.598212 10977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "4955defa995f4f5f8c510f1ad24c1464"
format_stamp: "Formatted at 2026-08-11 12:46:32 on dist-test-slave-f0tn"
server_key: "b95df198e2d9d8db851c8dc6e2cd4ed9"
server_key_iv: "3052a20663602b3bb2ae9eef4bd46162"
server_key_version: "encryptionkey@0"
I20260811 12:46:32.598400 10977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:32.612349 10977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:32.616365 10977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:32.616628 10977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:32.619342 10977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:32.619441 10977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:32.619607 10977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:32.619706 10977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:46:32 dist-test-slave-f0tn krb5kdc[10538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452392, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:32.643669 10774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:32.619761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:48881 (local address 127.7.111.126:32907)
0811 12:46:32.619959 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:32.619966 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:32.621434 (+ 1468us) server_negotiation.cc:407] Connection header received
0811 12:46:32.624252 (+ 2818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:32.624260 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:32.624357 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:32.624497 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:32.626933 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:32.627878 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:32.630348 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:32.630869 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:32.630979 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:32.635850 (+ 4871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:32.635858 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:32.635864 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:32.635938 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:32.639387 (+ 3449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:32.640519 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:32.640525 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:32.640528 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:32.640696 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:32.641402 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:32.641408 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:32.641412 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:32.641831 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:32.641985 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:32.643179 (+ 1194us) server_negotiation.cc:299] Negotiation successful
0811 12:46:32.643378 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:46:32.645221 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:32.620354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:32907 (local address 127.7.111.66:48881)
0811 12:46:32.621226 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:32.621285 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:32.623881 (+ 2596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:32.624779 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:32.624791 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:32.625683 (+ 892us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:32.626696 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:32.626745 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:32.628053 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:32.628059 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:32.630201 (+ 2142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:32.630218 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:32.631121 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:32.631129 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:32.631377 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:32.632118 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:32.632150 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:32.635490 (+ 3340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:32.639576 (+ 4086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:32.639589 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:32.639606 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:32.640326 (+ 720us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:32.640844 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:32.640848 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:32.640852 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:32.641196 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:32.642077 (+ 881us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:32.642087 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:32.642658 (+ 571us) client_negotiation.cc:769] Sending connection context
0811 12:46:32.643027 (+ 369us) client_negotiation.cc:240] Negotiation successful
0811 12:46:32.643640 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":341,"threads_started":1}
I20260811 12:46:32.666940 11143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:32.651092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:32907 (local address 127.7.111.66:52787)
0811 12:46:32.651650 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:32.651666 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:32.651804 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:32.652128 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:32.652132 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:32.652605 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:32.652900 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:32.652909 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:32.654073 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:32.654080 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:32.655789 (+ 1709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:32.655806 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:32.656588 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:32.656594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:32.656800 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:32.657708 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:32.657741 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:32.658846 (+ 1105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:32.662863 (+ 4017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:32.662869 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:32.662874 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:32.663622 (+ 748us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:32.664213 (+ 591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:32.664219 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:32.664223 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:32.664414 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:32.665350 (+ 936us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:32.665355 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:32.665640 (+ 285us) client_negotiation.cc:769] Sending connection context
0811 12:46:32.665828 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 12:46:32.666297 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":304,"threads_started":1}
I20260811 12:46:32.668047 10774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:32.650948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:52787 (local address 127.7.111.126:32907)
0811 12:46:32.651120 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:32.651124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:32.651760 (+ 636us) server_negotiation.cc:407] Connection header received
0811 12:46:32.651914 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:32.651920 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:32.652006 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:32.652087 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:32.653058 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:32.653897 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:32.655954 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:32.656400 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:32.656494 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:32.659076 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:32.659083 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:32.659087 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:32.659126 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:32.662678 (+ 3552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:32.663803 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:32.663810 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:32.663814 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:32.664020 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:32.664585 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:32.664591 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:32.664595 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:32.665123 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:32.665276 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:32.667436 (+ 2160us) server_negotiation.cc:299] Negotiation successful
0811 12:46:32.667707 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:46:32.671939 10977 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:42617
I20260811 12:46:32.671988 11216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:42617 every 8 connection(s)
I20260811 12:46:32.673401 10977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:46:32.675355 11218 heartbeater.cc:347] Connected to a master server at 127.7.111.126:32907
I20260811 12:46:32.675576 11218 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:32.676239 11218 heartbeater.cc:511] Master 127.7.111.126:32907 requested a full tablet report, sending...
I20260811 12:46:32.676720 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 10977
I20260811 12:46:32.676908 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:46:32.677482 7613 external_mini_cluster.cc:1428] Setting key 9377dbb2c8f3f2f1af36a7ecc8e764f3
I20260811 12:46:32.677834 10615 ts_manager.cc:194] Registered new tserver with Master: 4955defa995f4f5f8c510f1ad24c1464 (127.7.111.66:42617)
I20260811 12:46:32.678778 10615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:48881
WARNING: no policy specified for kudu/127.7.111.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
I20260811 12:46:32.722303 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:32907
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:34719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:33.051210 11226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:33.051486 11226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:33.051632 11226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:33.060842 11226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:33.061029 11226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:33.061204 11226 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 12:46:33.061301 11226 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:46:33.061606 11226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:46:33.072039 11226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:34719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.111.126:32907
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:33.073342 11226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:33.074724 11226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:33.088917 11232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:33.088918 11233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:33.089231 11238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:33.089702 11226 server_base.cc:1034] running on GCE node
I20260811 12:46:33.090370 11226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:33.091282 11226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:33.092485 11226 hybrid_clock.cc:648] HybridClock initialized: now 1786452393092428 us; error 50 us; skew 500 ppm
I20260811 12:46:33.092806 11226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:33 dist-test-slave-f0tn krb5kdc[10538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452393, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:33.096051 11226 init.cc:377] Logged in from keytab as kudu/127.7.111.67@KRBTEST.COM (short username kudu)
I20260811 12:46:33.097604 11226 webserver.cc:460] Webserver started at http://127.7.111.67:35345/ using document root <none> and password file <none>
I20260811 12:46:33.098268 11226 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:33.098368 11226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:33.098631 11226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:33.101553 11226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "e02640acffd14a7dad0476d33716e47f"
format_stamp: "Formatted at 2026-08-11 12:46:33 on dist-test-slave-f0tn"
server_key: "e543039f2010f51aa2853afe2e89f8cc"
server_key_iv: "5eb6b70417f71fc902b4f3bbad316556"
server_key_version: "encryptionkey@0"
I20260811 12:46:33.102300 11226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "e02640acffd14a7dad0476d33716e47f"
format_stamp: "Formatted at 2026-08-11 12:46:33 on dist-test-slave-f0tn"
server_key: "e543039f2010f51aa2853afe2e89f8cc"
server_key_iv: "5eb6b70417f71fc902b4f3bbad316556"
server_key_version: "encryptionkey@0"
I20260811 12:46:33.107271 11226 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 12:46:33.110607 11251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:33.111888 11226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:46:33.112033 11226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "e02640acffd14a7dad0476d33716e47f"
format_stamp: "Formatted at 2026-08-11 12:46:33 on dist-test-slave-f0tn"
server_key: "e543039f2010f51aa2853afe2e89f8cc"
server_key_iv: "5eb6b70417f71fc902b4f3bbad316556"
server_key_version: "encryptionkey@0"
I20260811 12:46:33.112237 11226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:33.124507 11226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:33.128551 11226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:33.128806 11226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:33.131503 11226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:33.131640 11226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:33.131798 11226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:33.131879 11226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:46:33 dist-test-slave-f0tn krb5kdc[10538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452393, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:33.154076 10774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.131929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:46949 (local address 127.7.111.126:32907)
0811 12:46:33.132118 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:33.132124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:33.133426 (+ 1302us) server_negotiation.cc:407] Connection header received
0811 12:46:33.135456 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:33.135462 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:33.135560 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:33.135735 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:33.138076 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.139172 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.141993 (+ 2821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.142571 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.142694 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.146967 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:33.146972 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:33.146975 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:33.147009 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:33.150493 (+ 3484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:33.151652 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:33.151656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:33.151658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:33.151804 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:33.152289 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:33.152292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:33.152294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:33.152548 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:33.152658 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:33.153622 (+ 964us) server_negotiation.cc:299] Negotiation successful
0811 12:46:33.153818 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 12:46:33.155720 11267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.132395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:32907 (local address 127.7.111.67:46949)
0811 12:46:33.133222 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:33.133267 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:33.135148 (+ 1881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:33.136090 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:33.136101 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:33.136956 (+ 855us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:33.137874 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.137905 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.139377 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.139382 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.141589 (+ 2207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.141607 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.142835 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.142844 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.143057 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.143740 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:33.143761 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:33.146690 (+ 2929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:33.150698 (+ 4008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:33.150715 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:33.150738 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:33.151497 (+ 759us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:33.151908 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:33.151914 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:33.151918 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:33.152193 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:33.152660 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:33.152669 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:33.153164 (+ 495us) client_negotiation.cc:769] Sending connection context
0811 12:46:33.153508 (+ 344us) client_negotiation.cc:240] Negotiation successful
0811 12:46:33.154118 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":331,"threads_started":1}
I20260811 12:46:33.175096 10970 heartbeater.cc:503] Master 127.7.111.126:32907 was elected leader, sending a full tablet report...
I20260811 12:46:33.182878 11389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.164051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:32907 (local address 127.7.111.67:34961)
0811 12:46:33.164684 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:33.164701 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:33.164826 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:33.165117 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:33.165121 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:33.165703 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:33.169537 (+ 3834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.169561 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.171002 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.171008 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.172784 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.172801 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.174050 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.174057 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.174245 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.174958 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:33.174975 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:33.175926 (+ 951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:33.179911 (+ 3985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:33.179916 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:33.179919 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:33.180563 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:33.181007 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:33.181015 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:33.181019 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:33.181190 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:33.181856 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:33.181862 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:33.182083 (+ 221us) client_negotiation.cc:769] Sending connection context
0811 12:46:33.182243 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 12:46:33.182497 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":332,"threads_started":1}
I20260811 12:46:33.182917 10774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.164004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:34961 (local address 127.7.111.126:32907)
0811 12:46:33.164170 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:33.164176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:33.164780 (+ 604us) server_negotiation.cc:407] Connection header received
0811 12:46:33.164934 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:33.164940 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:33.165003 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:33.165080 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:33.169719 (+ 4639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.170768 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.172969 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.173639 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.173745 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.176171 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:33.176178 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:33.176184 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:33.176224 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:33.179762 (+ 3538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:33.180706 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:33.180711 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:33.180715 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:33.180876 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:33.181301 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:33.181306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:33.181310 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:33.181716 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:33.181888 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:33.182354 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 12:46:33.182571 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:46:33.184125 11226 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:41823
I20260811 12:46:33.184188 11463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:41823 every 8 connection(s)
I20260811 12:46:33.185432 11226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:46:33.186887 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 11226
I20260811 12:46:33.187070 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:46:33.187497 11465 heartbeater.cc:347] Connected to a master server at 127.7.111.126:32907
I20260811 12:46:33.187618 7613 external_mini_cluster.cc:1428] Setting key cf6929b50a3adf3088af10d404a3d2e6
I20260811 12:46:33.187762 11465 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:33.188802 11465 heartbeater.cc:511] Master 127.7.111.126:32907 requested a full tablet report, sending...
I20260811 12:46:33.190205 10614 ts_manager.cc:194] Registered new tserver with Master: e02640acffd14a7dad0476d33716e47f (127.7.111.67:41823)
I20260811 12:46:33.191090 10614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:46949
I20260811 12:46:33.202935 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 12:46:33 dist-test-slave-f0tn krb5kdc[10538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 12:46:33 dist-test-slave-f0tn krb5kdc[10538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:33.257612 10774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.244187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51328 (local address 127.7.111.126:32907)
0811 12:46:33.244430 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:33.244437 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:33.244773 (+ 336us) server_negotiation.cc:407] Connection header received
0811 12:46:33.244956 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:33.244961 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:33.245030 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:33.245117 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:33.246647 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.247389 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.248634 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.249043 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.249162 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.252968 (+ 3806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:33.252973 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:33.252977 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:33.253012 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:33.255442 (+ 2430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:33.256008 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:33.256011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:33.256014 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:33.256131 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:33.256416 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:33.256419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:33.256421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:33.256684 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:33.256868 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:33.257060 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 12:46:33.257267 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 12:46:33.261580 10614 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51328:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:46:33.268353 10614 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:46:33.291735 11494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.280846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47046 (local address 127.7.111.67:41823)
0811 12:46:33.283827 (+ 2981us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:33.283835 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:33.283868 (+ 33us) server_negotiation.cc:407] Connection header received
0811 12:46:33.283963 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:33.283971 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:33.284216 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:33.284398 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:33.285735 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.287317 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.289765 (+ 2448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.290709 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.290834 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.291065 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:33.291307 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 12:46:33.291477 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2802,"thread_start_us":2614,"threads_started":1}
I20260811 12:46:33.292436 11493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.280475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:41823 (local address 127.0.0.1:47046)
0811 12:46:33.283222 (+ 2747us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:33.283238 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:33.283340 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:33.284724 (+ 1384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:33.284729 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:33.284791 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:33.285519 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.285534 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.287486 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.287491 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.289615 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.289628 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.290878 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.290883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.291005 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.291073 (+ 68us) client_negotiation.cc:769] Sending connection context
0811 12:46:33.291916 (+ 843us) client_negotiation.cc:240] Negotiation successful
0811 12:46:33.292107 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2630,"mutex_wait_us":232,"spinlock_wait_cycles":9984,"thread_start_us":2480,"threads_started":1}
I20260811 12:46:33.293172 11495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.283358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:36087 (local address 127.0.0.1:33566)
0811 12:46:33.285607 (+ 2249us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:33.285623 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:33.285727 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:33.286207 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:33.286212 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:33.286234 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:33.286562 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.286570 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.288043 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.288047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.290540 (+ 2493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.290556 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.292499 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.292505 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.292680 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.292721 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 12:46:33.292803 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 12:46:33.292917 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2131,"mutex_wait_us":99,"thread_start_us":1803,"threads_started":1}
I20260811 12:46:33.293815 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.283852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33566 (local address 127.7.111.65:36087)
0811 12:46:33.284967 (+ 1115us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:33.284975 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:33.285812 (+ 837us) server_negotiation.cc:407] Connection header received
0811 12:46:33.285892 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:33.285896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:33.286040 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:33.286189 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:33.286721 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.287900 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.290713 (+ 2813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.292279 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.292414 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.292595 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:33.293400 (+ 805us) server_negotiation.cc:299] Negotiation successful
0811 12:46:33.293547 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":959,"thread_start_us":822,"threads_started":1}
I20260811 12:46:33.294211 11490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.280283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:42617 (local address 127.0.0.1:42932)
0811 12:46:33.281320 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:33.281364 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:33.281565 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:33.284711 (+ 3146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:33.284719 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:33.284779 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:33.285500 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.285516 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.287941 (+ 2425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.287946 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.290540 (+ 2594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.290556 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.293548 (+ 2992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.293555 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.293721 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.293744 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 12:46:33.293798 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 12:46:33.293881 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":172,"spinlock_wait_cycles":2944,"thread_start_us":785,"threads_started":1}
I20260811 12:46:33.294937 11492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.280472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42932 (local address 127.7.111.66:42617)
0811 12:46:33.284080 (+ 3608us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:33.284089 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:33.284117 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:46:33.284230 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:33.284237 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:33.284466 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:33.285311 (+ 845us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:33.285735 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.287786 (+ 2051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.291168 (+ 3382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.293190 (+ 2022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.294414 (+ 1224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.294564 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:33.294615 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 12:46:33.294735 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3439,"thread_start_us":3888,"threads_started":1}
I20260811 12:46:33.296319 11333 tablet_service.cc:1467] Processing CreateTablet for tablet 648d9eb802644e709f7c77b5aaf6ed2c (DEFAULT_TABLE table=test-table [id=0a5f9057080e4da8866a250467c241cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:33.297771 11333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 648d9eb802644e709f7c77b5aaf6ed2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:33.298296 10843 tablet_service.cc:1467] Processing CreateTablet for tablet 648d9eb802644e709f7c77b5aaf6ed2c (DEFAULT_TABLE table=test-table [id=0a5f9057080e4da8866a250467c241cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:33.298655 11092 tablet_service.cc:1467] Processing CreateTablet for tablet 648d9eb802644e709f7c77b5aaf6ed2c (DEFAULT_TABLE table=test-table [id=0a5f9057080e4da8866a250467c241cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:33.299670 10843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 648d9eb802644e709f7c77b5aaf6ed2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:33.299901 11092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 648d9eb802644e709f7c77b5aaf6ed2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:33.310235 11498 tablet_bootstrap.cc:492] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f: Bootstrap starting.
I20260811 12:46:33.313145 11498 tablet_bootstrap.cc:654] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:33.314334 11504 tablet_bootstrap.cc:492] T 648d9eb802644e709f7c77b5aaf6ed2c P 6ff8914aa93c40f68d4a52bc8271e0c2: Bootstrap starting.
I20260811 12:46:33.314677 11498 log.cc:824] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f: Log is configured to *not* fsync() on all Append() calls
I20260811 12:46:33.315907 11502 tablet_bootstrap.cc:492] T 648d9eb802644e709f7c77b5aaf6ed2c P 4955defa995f4f5f8c510f1ad24c1464: Bootstrap starting.
I20260811 12:46:33.318578 11504 tablet_bootstrap.cc:654] T 648d9eb802644e709f7c77b5aaf6ed2c P 6ff8914aa93c40f68d4a52bc8271e0c2: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:33.319689 11502 tablet_bootstrap.cc:654] T 648d9eb802644e709f7c77b5aaf6ed2c P 4955defa995f4f5f8c510f1ad24c1464: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:33.319826 11498 tablet_bootstrap.cc:492] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f: No bootstrap required, opened a new log
I20260811 12:46:33.319847 11504 log.cc:824] T 648d9eb802644e709f7c77b5aaf6ed2c P 6ff8914aa93c40f68d4a52bc8271e0c2: Log is configured to *not* fsync() on all Append() calls
I20260811 12:46:33.320113 11498 ts_tablet_manager.cc:1397] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 12:46:33.321244 11502 log.cc:824] T 648d9eb802644e709f7c77b5aaf6ed2c P 4955defa995f4f5f8c510f1ad24c1464: Log is configured to *not* fsync() on all Append() calls
I20260811 12:46:33.322806 11504 tablet_bootstrap.cc:492] T 648d9eb802644e709f7c77b5aaf6ed2c P 6ff8914aa93c40f68d4a52bc8271e0c2: No bootstrap required, opened a new log
I20260811 12:46:33.323043 11504 ts_tablet_manager.cc:1397] T 648d9eb802644e709f7c77b5aaf6ed2c P 6ff8914aa93c40f68d4a52bc8271e0c2: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 12:46:33.324402 11502 tablet_bootstrap.cc:492] T 648d9eb802644e709f7c77b5aaf6ed2c P 4955defa995f4f5f8c510f1ad24c1464: No bootstrap required, opened a new log
I20260811 12:46:33.324674 11502 ts_tablet_manager.cc:1397] T 648d9eb802644e709f7c77b5aaf6ed2c P 4955defa995f4f5f8c510f1ad24c1464: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 12:46:33.331067 11504 raft_consensus.cc:348] T 648d9eb802644e709f7c77b5aaf6ed2c P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } } peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } }
I20260811 12:46:33.331125 11498 raft_consensus.cc:348] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } } peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } }
I20260811 12:46:33.331497 11504 raft_consensus.cc:374] T 648d9eb802644e709f7c77b5aaf6ed2c P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:33.331619 11504 raft_consensus.cc:729] T 648d9eb802644e709f7c77b5aaf6ed2c P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ff8914aa93c40f68d4a52bc8271e0c2, State: Initialized, Role: FOLLOWER
I20260811 12:46:33.331615 11498 raft_consensus.cc:374] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:33.331746 11498 raft_consensus.cc:729] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e02640acffd14a7dad0476d33716e47f, State: Initialized, Role: FOLLOWER
I20260811 12:46:33.332331 11504 consensus_queue.cc:260] T 648d9eb802644e709f7c77b5aaf6ed2c P 6ff8914aa93c40f68d4a52bc8271e0c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } } peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } }
I20260811 12:46:33.332405 11498 consensus_queue.cc:260] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } } peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } }
I20260811 12:46:33.334247 11504 ts_tablet_manager.cc:1428] T 648d9eb802644e709f7c77b5aaf6ed2c P 6ff8914aa93c40f68d4a52bc8271e0c2: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260811 12:46:33.335530 11465 heartbeater.cc:503] Master 127.7.111.126:32907 was elected leader, sending a full tablet report...
I20260811 12:46:33.336072 11498 ts_tablet_manager.cc:1428] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f: Time spent starting tablet: real 0.016s user 0.009s sys 0.008s
I20260811 12:46:33.336355 11502 raft_consensus.cc:348] T 648d9eb802644e709f7c77b5aaf6ed2c P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } } peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } }
I20260811 12:46:33.336793 11502 raft_consensus.cc:374] T 648d9eb802644e709f7c77b5aaf6ed2c P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:33.336930 11502 raft_consensus.cc:729] T 648d9eb802644e709f7c77b5aaf6ed2c P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4955defa995f4f5f8c510f1ad24c1464, State: Initialized, Role: FOLLOWER
I20260811 12:46:33.337594 11502 consensus_queue.cc:260] T 648d9eb802644e709f7c77b5aaf6ed2c P 4955defa995f4f5f8c510f1ad24c1464 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } } peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } }
I20260811 12:46:33.339107 11218 heartbeater.cc:503] Master 127.7.111.126:32907 was elected leader, sending a full tablet report...
I20260811 12:46:33.339506 11502 ts_tablet_manager.cc:1428] T 648d9eb802644e709f7c77b5aaf6ed2c P 4955defa995f4f5f8c510f1ad24c1464: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
W20260811 12:46:33.417979 10971 tablet.cc:2369] T 648d9eb802644e709f7c77b5aaf6ed2c P 6ff8914aa93c40f68d4a52bc8271e0c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:46:33.425606 11219 tablet.cc:2369] T 648d9eb802644e709f7c77b5aaf6ed2c P 4955defa995f4f5f8c510f1ad24c1464: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:46:33.439069 11466 tablet.cc:2369] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:46:33.483969 11510 raft_consensus.cc:482] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:46:33.484443 11510 raft_consensus.cc:504] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } } peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } }
I20260811 12:46:33.486192 11510 leader_election.cc:288] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4955defa995f4f5f8c510f1ad24c1464 (127.7.111.66:42617), 6ff8914aa93c40f68d4a52bc8271e0c2 (127.7.111.65:36087)
I20260811 12:46:33.492475 11267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.486521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:36087 (local address 127.7.111.67:54019)
0811 12:46:33.486756 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:33.486776 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:33.486886 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:33.487344 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:33.487348 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:33.487376 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:33.487620 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.487633 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.489215 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.489224 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.490608 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.490621 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.491773 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.491779 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.491968 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.491993 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 12:46:33.492079 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 12:46:33.492202 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 12:46:33.492524 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.486677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:54019 (local address 127.7.111.65:36087)
0811 12:46:33.487093 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:33.487098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:33.487113 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:46:33.487171 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:33.487174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:33.487231 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:33.487294 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:33.487826 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.489048 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.490773 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.491621 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.491773 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.491848 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:33.492120 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 12:46:33.492259 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284}
I20260811 12:46:33.493770 10880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "648d9eb802644e709f7c77b5aaf6ed2c" candidate_uuid: "e02640acffd14a7dad0476d33716e47f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" is_pre_election: true
I20260811 12:46:33.494481 10880 raft_consensus.cc:2454] T 648d9eb802644e709f7c77b5aaf6ed2c P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e02640acffd14a7dad0476d33716e47f in term 0.
I20260811 12:46:33.495270 11255 leader_election.cc:302] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ff8914aa93c40f68d4a52bc8271e0c2, e02640acffd14a7dad0476d33716e47f; no voters:
I20260811 12:46:33.495513 11492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.486602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:33739 (local address 127.7.111.66:42617)
0811 12:46:33.486829 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:33.486835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:33.488512 (+ 1677us) server_negotiation.cc:407] Connection header received
0811 12:46:33.488669 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:33.488674 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:33.488745 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:33.488819 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:33.489531 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.490633 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.493232 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.494491 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.494661 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.494729 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:33.495088 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 12:46:33.495235 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 12:46:33.495553 11512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.487608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:42617 (local address 127.7.111.67:33739)
0811 12:46:33.488411 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:33.488425 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:33.488545 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:33.488846 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:33.488850 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:33.488874 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:33.489305 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.489321 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.490814 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.490820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.493065 (+ 2245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.493082 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.494707 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.494713 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.494883 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.494918 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 12:46:33.495045 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 12:46:33.495172 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"thread_start_us":499,"threads_started":1}
I20260811 12:46:33.495777 11510 raft_consensus.cc:2781] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:46:33.495883 11510 raft_consensus.cc:482] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:46:33.495946 11510 raft_consensus.cc:3037] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:33.498368 11510 raft_consensus.cc:504] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } } peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } }
I20260811 12:46:33.498974 11510 leader_election.cc:288] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f [CANDIDATE]: Term 1 election: Requested vote from peers 4955defa995f4f5f8c510f1ad24c1464 (127.7.111.66:42617), 6ff8914aa93c40f68d4a52bc8271e0c2 (127.7.111.65:36087)
I20260811 12:46:33.499023 11137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "648d9eb802644e709f7c77b5aaf6ed2c" candidate_uuid: "e02640acffd14a7dad0476d33716e47f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4955defa995f4f5f8c510f1ad24c1464" is_pre_election: true
I20260811 12:46:33.499482 10880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "648d9eb802644e709f7c77b5aaf6ed2c" candidate_uuid: "e02640acffd14a7dad0476d33716e47f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2"
I20260811 12:46:33.499487 11135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "648d9eb802644e709f7c77b5aaf6ed2c" candidate_uuid: "e02640acffd14a7dad0476d33716e47f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4955defa995f4f5f8c510f1ad24c1464"
I20260811 12:46:33.499686 10880 raft_consensus.cc:3037] T 648d9eb802644e709f7c77b5aaf6ed2c P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:33.499717 11137 raft_consensus.cc:2454] T 648d9eb802644e709f7c77b5aaf6ed2c P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e02640acffd14a7dad0476d33716e47f in term 0.
I20260811 12:46:33.502189 10880 raft_consensus.cc:2454] T 648d9eb802644e709f7c77b5aaf6ed2c P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e02640acffd14a7dad0476d33716e47f in term 1.
I20260811 12:46:33.502539 11255 leader_election.cc:302] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ff8914aa93c40f68d4a52bc8271e0c2, e02640acffd14a7dad0476d33716e47f; no voters: 4955defa995f4f5f8c510f1ad24c1464
I20260811 12:46:33.502933 11510 raft_consensus.cc:2781] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:46:33.503510 11510 raft_consensus.cc:686] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f [term 1 LEADER]: Becoming Leader. State: Replica: e02640acffd14a7dad0476d33716e47f, State: Running, Role: LEADER
I20260811 12:46:33.504045 11510 consensus_queue.cc:237] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } } peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } }
I20260811 12:46:33.508720 10614 catalog_manager.cc:5495] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f reported cstate change: term changed from 0 to 1, leader changed from <none> to e02640acffd14a7dad0476d33716e47f (127.7.111.67). New cstate: current_term: 1 leader_uuid: "e02640acffd14a7dad0476d33716e47f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } health_report { overall_health: UNKNOWN } } }
Aug 11 12:46:33 dist-test-slave-f0tn krb5kdc[10538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452391, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:33.556666 11524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.543920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:32907 (local address 127.0.0.1:51340)
0811 12:46:33.544579 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:33.544596 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:33.544733 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:33.545030 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:33.545033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:33.545873 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:33.546164 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.546175 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.547202 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.547206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.548275 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.548288 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.549002 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.549006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.549125 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.549729 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:33.549746 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:33.551884 (+ 2138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:33.554441 (+ 2557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:33.554451 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:33.554457 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:33.554874 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:33.555189 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:33.555192 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:33.555195 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:33.555345 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:33.555837 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:33.555843 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:33.555978 (+ 135us) client_negotiation.cc:769] Sending connection context
0811 12:46:33.556111 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 12:46:33.556300 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":371,"threads_started":1}
I20260811 12:46:33.556753 10774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.543827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51340 (local address 127.7.111.126:32907)
0811 12:46:33.543985 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:33.543989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:33.544674 (+ 685us) server_negotiation.cc:407] Connection header received
0811 12:46:33.544837 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:33.544841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:33.544913 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:33.544985 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:33.546309 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.547099 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.548385 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.548848 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.548939 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.552035 (+ 3096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:33.552040 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:33.552044 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:33.552072 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:33.554323 (+ 2251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:33.554966 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:33.554969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:33.554972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:33.555099 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:33.555431 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:33.555434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:33.555436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:33.555728 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:33.555832 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:33.556199 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 12:46:33.556450 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":54}
I20260811 12:46:33.561406 10614 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:51340:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 12:46:33.562381 10614 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:46:33.568562 10843 tablet_service.cc:1467] Processing CreateTablet for tablet 58c1ad8738ac42f7ba1ed91c827015ea (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=374a6b3116c941cf8de108ec06c9a8cc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:46:33.568679 11092 tablet_service.cc:1467] Processing CreateTablet for tablet 58c1ad8738ac42f7ba1ed91c827015ea (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=374a6b3116c941cf8de108ec06c9a8cc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:46:33.568691 11333 tablet_service.cc:1467] Processing CreateTablet for tablet 58c1ad8738ac42f7ba1ed91c827015ea (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=374a6b3116c941cf8de108ec06c9a8cc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:46:33.569612 11092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58c1ad8738ac42f7ba1ed91c827015ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:33.569612 11333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58c1ad8738ac42f7ba1ed91c827015ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:33.569629 10843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58c1ad8738ac42f7ba1ed91c827015ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:33.576131 11504 tablet_bootstrap.cc:492] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2: Bootstrap starting.
I20260811 12:46:33.576956 11502 tablet_bootstrap.cc:492] T 58c1ad8738ac42f7ba1ed91c827015ea P 4955defa995f4f5f8c510f1ad24c1464: Bootstrap starting.
I20260811 12:46:33.577589 11498 tablet_bootstrap.cc:492] T 58c1ad8738ac42f7ba1ed91c827015ea P e02640acffd14a7dad0476d33716e47f: Bootstrap starting.
I20260811 12:46:33.577879 11504 tablet_bootstrap.cc:654] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:33.578749 11502 tablet_bootstrap.cc:654] T 58c1ad8738ac42f7ba1ed91c827015ea P 4955defa995f4f5f8c510f1ad24c1464: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:33.579010 11498 tablet_bootstrap.cc:654] T 58c1ad8738ac42f7ba1ed91c827015ea P e02640acffd14a7dad0476d33716e47f: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:33.580631 11504 tablet_bootstrap.cc:492] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2: No bootstrap required, opened a new log
I20260811 12:46:33.580789 11504 ts_tablet_manager.cc:1397] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 12:46:33.581230 11502 tablet_bootstrap.cc:492] T 58c1ad8738ac42f7ba1ed91c827015ea P 4955defa995f4f5f8c510f1ad24c1464: No bootstrap required, opened a new log
I20260811 12:46:33.581409 11502 ts_tablet_manager.cc:1397] T 58c1ad8738ac42f7ba1ed91c827015ea P 4955defa995f4f5f8c510f1ad24c1464: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 12:46:33.581596 11498 tablet_bootstrap.cc:492] T 58c1ad8738ac42f7ba1ed91c827015ea P e02640acffd14a7dad0476d33716e47f: No bootstrap required, opened a new log
I20260811 12:46:33.581532 11504 raft_consensus.cc:348] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:33.581755 11498 ts_tablet_manager.cc:1397] T 58c1ad8738ac42f7ba1ed91c827015ea P e02640acffd14a7dad0476d33716e47f: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260811 12:46:33.581770 11504 raft_consensus.cc:374] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:33.581862 11504 raft_consensus.cc:729] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ff8914aa93c40f68d4a52bc8271e0c2, State: Initialized, Role: FOLLOWER
I20260811 12:46:33.582088 11504 consensus_queue.cc:260] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:33.582252 11502 raft_consensus.cc:348] T 58c1ad8738ac42f7ba1ed91c827015ea P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:33.582494 11502 raft_consensus.cc:374] T 58c1ad8738ac42f7ba1ed91c827015ea P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:33.582621 11502 raft_consensus.cc:729] T 58c1ad8738ac42f7ba1ed91c827015ea P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4955defa995f4f5f8c510f1ad24c1464, State: Initialized, Role: FOLLOWER
I20260811 12:46:33.582521 11498 raft_consensus.cc:348] T 58c1ad8738ac42f7ba1ed91c827015ea P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:33.582741 11498 raft_consensus.cc:374] T 58c1ad8738ac42f7ba1ed91c827015ea P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:33.582510 11508 tablet_replica.cc:439] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } } }
I20260811 12:46:33.582870 11508 tablet_replica.cc:442] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:46:33.582875 11498 raft_consensus.cc:729] T 58c1ad8738ac42f7ba1ed91c827015ea P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e02640acffd14a7dad0476d33716e47f, State: Initialized, Role: FOLLOWER
I20260811 12:46:33.582856 11502 consensus_queue.cc:260] T 58c1ad8738ac42f7ba1ed91c827015ea P 4955defa995f4f5f8c510f1ad24c1464 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:33.583096 11498 consensus_queue.cc:260] T 58c1ad8738ac42f7ba1ed91c827015ea P e02640acffd14a7dad0476d33716e47f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:33.583518 11502 ts_tablet_manager.cc:1428] T 58c1ad8738ac42f7ba1ed91c827015ea P 4955defa995f4f5f8c510f1ad24c1464: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 12:46:33.583652 11498 ts_tablet_manager.cc:1428] T 58c1ad8738ac42f7ba1ed91c827015ea P e02640acffd14a7dad0476d33716e47f: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 12:46:33.583314 11511 tablet_replica.cc:439] T 58c1ad8738ac42f7ba1ed91c827015ea P 4955defa995f4f5f8c510f1ad24c1464: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } } }
I20260811 12:46:33.583817 11511 tablet_replica.cc:442] T 58c1ad8738ac42f7ba1ed91c827015ea P 4955defa995f4f5f8c510f1ad24c1464: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:46:33.583902 11510 tablet_replica.cc:439] T 58c1ad8738ac42f7ba1ed91c827015ea P e02640acffd14a7dad0476d33716e47f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } } }
I20260811 12:46:33.584102 11510 tablet_replica.cc:442] T 58c1ad8738ac42f7ba1ed91c827015ea P e02640acffd14a7dad0476d33716e47f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:46:33.585750 11504 ts_tablet_manager.cc:1428] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260811 12:46:33.589476 11508 raft_consensus.cc:482] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:46:33.589641 11508 raft_consensus.cc:504] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:33.591136 11508 leader_election.cc:288] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4955defa995f4f5f8c510f1ad24c1464 (127.7.111.66:42617), e02640acffd14a7dad0476d33716e47f (127.7.111.67:41823)
I20260811 12:46:33.599504 11492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.591557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:52953 (local address 127.7.111.66:42617)
0811 12:46:33.591758 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:33.591764 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:33.592247 (+ 483us) server_negotiation.cc:407] Connection header received
0811 12:46:33.592488 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:33.592493 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:33.592569 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:33.592650 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:33.593395 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.594539 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.597168 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.598434 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.598591 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.598665 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:33.599092 (+ 427us) server_negotiation.cc:299] Negotiation successful
0811 12:46:33.599236 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:46:33.599534 11494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.591906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:57831 (local address 127.7.111.67:41823)
0811 12:46:33.592081 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:33.592086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:33.592787 (+ 701us) server_negotiation.cc:407] Connection header received
0811 12:46:33.592944 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:33.592948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:33.592999 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:33.593064 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:33.593759 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.594798 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.597160 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.598434 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.598606 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.598685 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:33.599092 (+ 407us) server_negotiation.cc:299] Negotiation successful
0811 12:46:33.599220 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:46:33.599781 11536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.591708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:41823 (local address 127.7.111.65:57831)
0811 12:46:33.592667 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:33.592683 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:33.592826 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:33.593111 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:33.593116 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:33.593142 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:33.593595 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.593605 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.594913 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.594918 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.597025 (+ 2107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.597034 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.598662 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.598668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.598934 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.598984 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 12:46:33.599068 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 12:46:33.599258 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":842,"mutex_wait_us":121,"thread_start_us":625,"threads_started":1}
I20260811 12:46:33.599825 11533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.591409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:42617 (local address 127.7.111.65:52953)
0811 12:46:33.592118 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:33.592144 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:33.592343 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:33.592697 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:33.592701 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:33.592766 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:33.593141 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.593152 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.594658 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.594661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.597006 (+ 2345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.597022 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.598694 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.598700 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.598925 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.598974 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 12:46:33.599068 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 12:46:33.599258 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":65,"thread_start_us":378,"threads_started":1}
I20260811 12:46:33.600795 11385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58c1ad8738ac42f7ba1ed91c827015ea" candidate_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e02640acffd14a7dad0476d33716e47f" is_pre_election: true
I20260811 12:46:33.600977 11135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58c1ad8738ac42f7ba1ed91c827015ea" candidate_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4955defa995f4f5f8c510f1ad24c1464" is_pre_election: true
I20260811 12:46:33.601248 11385 raft_consensus.cc:2454] T 58c1ad8738ac42f7ba1ed91c827015ea P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ff8914aa93c40f68d4a52bc8271e0c2 in term 0.
I20260811 12:46:33.601326 11135 raft_consensus.cc:2454] T 58c1ad8738ac42f7ba1ed91c827015ea P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ff8914aa93c40f68d4a52bc8271e0c2 in term 0.
I20260811 12:46:33.601900 10764 leader_election.cc:302] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4955defa995f4f5f8c510f1ad24c1464, 6ff8914aa93c40f68d4a52bc8271e0c2; no voters:
I20260811 12:46:33.602337 11508 raft_consensus.cc:2781] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:46:33.602487 11508 raft_consensus.cc:482] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:46:33.602651 11508 raft_consensus.cc:3037] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:33.605022 11508 raft_consensus.cc:504] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:33.605798 11508 leader_election.cc:288] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2 [CANDIDATE]: Term 1 election: Requested vote from peers 4955defa995f4f5f8c510f1ad24c1464 (127.7.111.66:42617), e02640acffd14a7dad0476d33716e47f (127.7.111.67:41823)
I20260811 12:46:33.606190 11135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58c1ad8738ac42f7ba1ed91c827015ea" candidate_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4955defa995f4f5f8c510f1ad24c1464"
I20260811 12:46:33.606288 11385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58c1ad8738ac42f7ba1ed91c827015ea" candidate_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e02640acffd14a7dad0476d33716e47f"
I20260811 12:46:33.606405 11135 raft_consensus.cc:3037] T 58c1ad8738ac42f7ba1ed91c827015ea P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:33.606508 11385 raft_consensus.cc:3037] T 58c1ad8738ac42f7ba1ed91c827015ea P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:33.608773 11135 raft_consensus.cc:2454] T 58c1ad8738ac42f7ba1ed91c827015ea P 4955defa995f4f5f8c510f1ad24c1464 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ff8914aa93c40f68d4a52bc8271e0c2 in term 1.
I20260811 12:46:33.608824 11385 raft_consensus.cc:2454] T 58c1ad8738ac42f7ba1ed91c827015ea P e02640acffd14a7dad0476d33716e47f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ff8914aa93c40f68d4a52bc8271e0c2 in term 1.
I20260811 12:46:33.609151 10764 leader_election.cc:302] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4955defa995f4f5f8c510f1ad24c1464, 6ff8914aa93c40f68d4a52bc8271e0c2; no voters:
I20260811 12:46:33.609546 11508 raft_consensus.cc:2781] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:46:33.610138 11508 raft_consensus.cc:686] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 1 LEADER]: Becoming Leader. State: Replica: 6ff8914aa93c40f68d4a52bc8271e0c2, State: Running, Role: LEADER
I20260811 12:46:33.610910 11508 consensus_queue.cc:237] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:33.614023 11538 tablet_replica.cc:439] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2: TxnStatusTablet state changed. Reason: New leader 6ff8914aa93c40f68d4a52bc8271e0c2. Latest consensus state: current_term: 1 leader_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } } }
I20260811 12:46:33.614504 11538 tablet_replica.cc:442] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:46:33.615391 11541 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 12:46:33.615666 11541 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 12:46:33.616130 10613 catalog_manager.cc:5495] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ff8914aa93c40f68d4a52bc8271e0c2 (127.7.111.65). New cstate: current_term: 1 leader_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } health_report { overall_health: UNKNOWN } } }
I20260811 12:46:33.638652 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.632968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33578 (local address 127.7.111.65:36087)
0811 12:46:33.633317 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:33.633326 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:33.633345 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:46:33.633612 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:33.633616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:33.633696 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:33.633819 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:46:33.634208 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.635065 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.636398 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.636939 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.637056 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.637521 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:46:33.637985 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:46:33.638117 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:33.638237 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 12:46:33.638321 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231}
I20260811 12:46:33.638693 11524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.632954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:36087 (local address 127.0.0.1:33578)
0811 12:46:33.633178 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:33.633199 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:33.633340 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:33.633856 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:33.633861 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:33.633873 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:46:33.634071 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.634080 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.635189 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.635195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.636268 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.636280 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.637086 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.637092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.637257 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.637311 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:46:33.638093 (+ 782us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:46:33.638114 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 12:46:33.638193 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 12:46:33.638341 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 12:46:33.646557 11135 raft_consensus.cc:1261] T 58c1ad8738ac42f7ba1ed91c827015ea P 4955defa995f4f5f8c510f1ad24c1464 [term 1 FOLLOWER]: Refusing update from remote peer 6ff8914aa93c40f68d4a52bc8271e0c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:46:33.646740 11385 raft_consensus.cc:1261] T 58c1ad8738ac42f7ba1ed91c827015ea P e02640acffd14a7dad0476d33716e47f [term 1 FOLLOWER]: Refusing update from remote peer 6ff8914aa93c40f68d4a52bc8271e0c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:46:33.647676 11538 consensus_queue.cc:1035] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:46:33.647985 11508 consensus_queue.cc:1035] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:46:33.658600 11511 tablet_replica.cc:439] T 58c1ad8738ac42f7ba1ed91c827015ea P 4955defa995f4f5f8c510f1ad24c1464: TxnStatusTablet state changed. Reason: New leader 6ff8914aa93c40f68d4a52bc8271e0c2. Latest consensus state: current_term: 1 leader_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } } }
I20260811 12:46:33.659013 11511 tablet_replica.cc:442] T 58c1ad8738ac42f7ba1ed91c827015ea P 4955defa995f4f5f8c510f1ad24c1464: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:46:33.662762 11550 mvcc.cc:205] Tried to move back new op lower bound from 7317309004366774272 to 7317309004233846784. Current Snapshot: MvccSnapshot[applied={T|T < 7317309004366774272}]
I20260811 12:46:33.664074 11561 tablet_replica.cc:439] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } } }
I20260811 12:46:33.665362 11561 tablet_replica.cc:442] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:46:33.666807 11511 tablet_replica.cc:439] T 58c1ad8738ac42f7ba1ed91c827015ea P 4955defa995f4f5f8c510f1ad24c1464: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } } }
I20260811 12:46:33.667065 11511 tablet_replica.cc:442] T 58c1ad8738ac42f7ba1ed91c827015ea P 4955defa995f4f5f8c510f1ad24c1464: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:46:33.671465 11510 tablet_replica.cc:439] T 58c1ad8738ac42f7ba1ed91c827015ea P e02640acffd14a7dad0476d33716e47f: TxnStatusTablet state changed. Reason: New leader 6ff8914aa93c40f68d4a52bc8271e0c2. Latest consensus state: current_term: 1 leader_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } } }
I20260811 12:46:33.671823 11510 tablet_replica.cc:442] T 58c1ad8738ac42f7ba1ed91c827015ea P e02640acffd14a7dad0476d33716e47f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:46:33.673283 11508 tablet_replica.cc:439] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } } }
I20260811 12:46:33.673635 11508 tablet_replica.cc:442] T 58c1ad8738ac42f7ba1ed91c827015ea P 6ff8914aa93c40f68d4a52bc8271e0c2: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:46:33.681982 11510 tablet_replica.cc:439] T 58c1ad8738ac42f7ba1ed91c827015ea P e02640acffd14a7dad0476d33716e47f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } } }
I20260811 12:46:33.682315 11510 tablet_replica.cc:442] T 58c1ad8738ac42f7ba1ed91c827015ea P e02640acffd14a7dad0476d33716e47f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:46:33.683669 11494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.677599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47054 (local address 127.7.111.67:41823)
0811 12:46:33.677768 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:33.677774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:33.677790 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:46:33.677864 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:33.677868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:33.677935 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:33.678055 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:46:33.678438 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.679307 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.680675 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.681210 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.681320 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.681901 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:46:33.682594 (+ 693us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:46:33.682721 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:33.683254 (+ 533us) server_negotiation.cc:299] Negotiation successful
0811 12:46:33.683345 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 12:46:33.705092 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.695651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:36169 (local address 127.7.111.65:36087)
0811 12:46:33.695919 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:33.695926 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:33.698901 (+ 2975us) server_negotiation.cc:407] Connection header received
0811 12:46:33.699095 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:33.699100 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:33.699187 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:33.699301 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:46:33.699930 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.701124 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.703305 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.703854 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.703948 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.704490 (+ 542us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:46:33.704587 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:46:33.704648 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:33.704794 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 12:46:33.704856 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260811 12:46:33.705305 11576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.695557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:36087 (local address 127.7.111.67:36169)
0811 12:46:33.698788 (+ 3231us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:33.698808 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:33.698962 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:33.699331 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:33.699336 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:33.699349 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:46:33.699741 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.699755 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.701298 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.701304 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.703162 (+ 1858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.703179 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.704029 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.704036 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.704226 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.704300 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:46:33.704679 (+ 379us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:46:33.704690 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 12:46:33.704772 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 12:46:33.704914 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3054,"thread_start_us":2870,"threads_started":1}
I20260811 12:46:33.715257 11135 raft_consensus.cc:3037] T 648d9eb802644e709f7c77b5aaf6ed2c P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:33.715600 10880 raft_consensus.cc:1261] T 648d9eb802644e709f7c77b5aaf6ed2c P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 1 FOLLOWER]: Refusing update from remote peer e02640acffd14a7dad0476d33716e47f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:46:33.716379 11510 consensus_queue.cc:1035] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:46:33.718780 11135 raft_consensus.cc:1261] T 648d9eb802644e709f7c77b5aaf6ed2c P 4955defa995f4f5f8c510f1ad24c1464 [term 1 FOLLOWER]: Refusing update from remote peer e02640acffd14a7dad0476d33716e47f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:46:33.719475 11510 consensus_queue.cc:1035] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:46:33.724704 11554 mvcc.cc:205] Tried to move back new op lower bound from 7317309004651741184 to 7317309003795996672. Current Snapshot: MvccSnapshot[applied={T|T < 7317309004651741184}]
I20260811 12:46:33.758839 11494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.750869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:57945 (local address 127.7.111.67:41823)
0811 12:46:33.751005 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:33.751010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:33.751915 (+ 905us) server_negotiation.cc:407] Connection header received
0811 12:46:33.752208 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:33.752215 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:33.752299 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:33.752396 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:46:33.753229 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.754227 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.756631 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.757239 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.757409 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.758088 (+ 679us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:46:33.758219 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:46:33.758311 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:33.758503 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 12:46:33.758639 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 12:46:33.759079 11586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:33.750942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:41823 (local address 127.7.111.65:57945)
0811 12:46:33.751796 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:33.751818 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:33.752035 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:33.752417 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:33.752422 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:33.752435 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:46:33.752961 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.752974 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.754428 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.754433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.756456 (+ 2023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:33.756476 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:33.757599 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:33.757606 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:33.757799 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:33.757882 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:46:33.758316 (+ 434us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:46:33.758337 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 12:46:33.758471 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 12:46:33.758664 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":553,"threads_started":1}
I20260811 12:46:33.804438 10611 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:46:33.804739 10611 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:46:33.809024 10611 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706: Sending DeleteTablet for 3 replicas of tablet 648d9eb802644e709f7c77b5aaf6ed2c
I20260811 12:46:33.810281 11092 tablet_service.cc:1514] Processing DeleteTablet for tablet 648d9eb802644e709f7c77b5aaf6ed2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:33 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:42932
I20260811 12:46:33.811368 11333 tablet_service.cc:1514] Processing DeleteTablet for tablet 648d9eb802644e709f7c77b5aaf6ed2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:33 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:47046
I20260811 12:46:33.812752 10843 tablet_service.cc:1514] Processing DeleteTablet for tablet 648d9eb802644e709f7c77b5aaf6ed2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:33 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:33566
I20260811 12:46:33.819664 10611 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51328:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:46:33.820389 10611 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:46:33.827936 10841 tablet_service.cc:1467] Processing CreateTablet for tablet 9efd2a3b86ac4cfd99a6f7240f268c19 (DEFAULT_TABLE table=test-table [id=24b65652297948f3a5c8aac1937547bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:33.828533 10841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9efd2a3b86ac4cfd99a6f7240f268c19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:33.828418 11332 tablet_service.cc:1467] Processing CreateTablet for tablet 9efd2a3b86ac4cfd99a6f7240f268c19 (DEFAULT_TABLE table=test-table [id=24b65652297948f3a5c8aac1937547bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:33.828960 11332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9efd2a3b86ac4cfd99a6f7240f268c19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:33.829855 11090 tablet_service.cc:1467] Processing CreateTablet for tablet 9efd2a3b86ac4cfd99a6f7240f268c19 (DEFAULT_TABLE table=test-table [id=24b65652297948f3a5c8aac1937547bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:33.830428 11090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9efd2a3b86ac4cfd99a6f7240f268c19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:33.835322 11504 tablet_bootstrap.cc:492] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2: Bootstrap starting.
I20260811 12:46:33.837132 11504 tablet_bootstrap.cc:654] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:33.842727 11589 tablet_replica.cc:330] T 648d9eb802644e709f7c77b5aaf6ed2c P 4955defa995f4f5f8c510f1ad24c1464: stopping tablet replica
I20260811 12:46:33.843657 11589 raft_consensus.cc:2229] T 648d9eb802644e709f7c77b5aaf6ed2c P 4955defa995f4f5f8c510f1ad24c1464 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:46:33.844295 11589 raft_consensus.cc:2258] T 648d9eb802644e709f7c77b5aaf6ed2c P 4955defa995f4f5f8c510f1ad24c1464 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:46:33.847822 11589 ts_tablet_manager.cc:1905] T 648d9eb802644e709f7c77b5aaf6ed2c P 4955defa995f4f5f8c510f1ad24c1464: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:46:33.851130 11502 tablet_bootstrap.cc:492] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 4955defa995f4f5f8c510f1ad24c1464: Bootstrap starting.
I20260811 12:46:33.853159 11502 tablet_bootstrap.cc:654] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 4955defa995f4f5f8c510f1ad24c1464: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:33.859025 11589 ts_tablet_manager.cc:1918] T 648d9eb802644e709f7c77b5aaf6ed2c P 4955defa995f4f5f8c510f1ad24c1464: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:46:33.859364 11589 log.cc:1196] T 648d9eb802644e709f7c77b5aaf6ed2c P 4955defa995f4f5f8c510f1ad24c1464: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/648d9eb802644e709f7c77b5aaf6ed2c
I20260811 12:46:33.860093 11589 ts_tablet_manager.cc:1939] T 648d9eb802644e709f7c77b5aaf6ed2c P 4955defa995f4f5f8c510f1ad24c1464: Deleting consensus metadata
I20260811 12:46:33.862375 10595 catalog_manager.cc:4841] TS 4955defa995f4f5f8c510f1ad24c1464 (127.7.111.66:42617): tablet 648d9eb802644e709f7c77b5aaf6ed2c (table test-table [id=0a5f9057080e4da8866a250467c241cb]) successfully deleted
I20260811 12:46:33.863914 11498 tablet_bootstrap.cc:492] T 9efd2a3b86ac4cfd99a6f7240f268c19 P e02640acffd14a7dad0476d33716e47f: Bootstrap starting.
I20260811 12:46:33.866082 11498 tablet_bootstrap.cc:654] T 9efd2a3b86ac4cfd99a6f7240f268c19 P e02640acffd14a7dad0476d33716e47f: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:33.868726 11498 tablet_bootstrap.cc:492] T 9efd2a3b86ac4cfd99a6f7240f268c19 P e02640acffd14a7dad0476d33716e47f: No bootstrap required, opened a new log
I20260811 12:46:33.869004 11498 ts_tablet_manager.cc:1397] T 9efd2a3b86ac4cfd99a6f7240f268c19 P e02640acffd14a7dad0476d33716e47f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 12:46:33.870280 11498 raft_consensus.cc:348] T 9efd2a3b86ac4cfd99a6f7240f268c19 P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:33.870748 11498 raft_consensus.cc:374] T 9efd2a3b86ac4cfd99a6f7240f268c19 P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:33.870903 11498 raft_consensus.cc:729] T 9efd2a3b86ac4cfd99a6f7240f268c19 P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e02640acffd14a7dad0476d33716e47f, State: Initialized, Role: FOLLOWER
I20260811 12:46:33.871260 11498 consensus_queue.cc:260] T 9efd2a3b86ac4cfd99a6f7240f268c19 P e02640acffd14a7dad0476d33716e47f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:33.873629 11498 ts_tablet_manager.cc:1428] T 9efd2a3b86ac4cfd99a6f7240f268c19 P e02640acffd14a7dad0476d33716e47f: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260811 12:46:33.880645 11504 tablet_bootstrap.cc:492] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2: No bootstrap required, opened a new log
I20260811 12:46:33.880893 11504 ts_tablet_manager.cc:1397] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2: Time spent bootstrapping tablet: real 0.046s user 0.007s sys 0.004s
I20260811 12:46:33.881830 11504 raft_consensus.cc:348] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:33.882093 11504 raft_consensus.cc:374] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:33.882218 11504 raft_consensus.cc:729] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ff8914aa93c40f68d4a52bc8271e0c2, State: Initialized, Role: FOLLOWER
I20260811 12:46:33.882833 11502 tablet_bootstrap.cc:492] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 4955defa995f4f5f8c510f1ad24c1464: No bootstrap required, opened a new log
I20260811 12:46:33.884049 11593 tablet_replica.cc:330] T 648d9eb802644e709f7c77b5aaf6ed2c P 6ff8914aa93c40f68d4a52bc8271e0c2: stopping tablet replica
I20260811 12:46:33.884547 11593 raft_consensus.cc:2229] T 648d9eb802644e709f7c77b5aaf6ed2c P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:46:33.884903 11593 raft_consensus.cc:2258] T 648d9eb802644e709f7c77b5aaf6ed2c P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:46:33.885274 11502 ts_tablet_manager.cc:1397] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 4955defa995f4f5f8c510f1ad24c1464: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.008s
I20260811 12:46:33.886122 11502 raft_consensus.cc:348] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:33.886371 11502 raft_consensus.cc:374] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:33.886448 11502 raft_consensus.cc:729] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4955defa995f4f5f8c510f1ad24c1464, State: Initialized, Role: FOLLOWER
I20260811 12:46:33.887216 11591 tablet_replica.cc:330] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f: stopping tablet replica
I20260811 12:46:33.887717 11591 raft_consensus.cc:2229] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:46:33.888336 11591 raft_consensus.cc:2258] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:46:33.887595 11504 consensus_queue.cc:260] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:33.888780 11502 consensus_queue.cc:260] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 4955defa995f4f5f8c510f1ad24c1464 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:33.889566 11502 ts_tablet_manager.cc:1428] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 4955defa995f4f5f8c510f1ad24c1464: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 12:46:33.889781 11504 ts_tablet_manager.cc:1428] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20260811 12:46:33.890571 11593 ts_tablet_manager.cc:1905] T 648d9eb802644e709f7c77b5aaf6ed2c P 6ff8914aa93c40f68d4a52bc8271e0c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:46:33.899310 11593 ts_tablet_manager.cc:1918] T 648d9eb802644e709f7c77b5aaf6ed2c P 6ff8914aa93c40f68d4a52bc8271e0c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:46:33.899472 11591 ts_tablet_manager.cc:1905] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:46:33.899645 11593 log.cc:1196] T 648d9eb802644e709f7c77b5aaf6ed2c P 6ff8914aa93c40f68d4a52bc8271e0c2: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/648d9eb802644e709f7c77b5aaf6ed2c
I20260811 12:46:33.900393 11593 ts_tablet_manager.cc:1939] T 648d9eb802644e709f7c77b5aaf6ed2c P 6ff8914aa93c40f68d4a52bc8271e0c2: Deleting consensus metadata
I20260811 12:46:33.902451 10592 catalog_manager.cc:4841] TS 6ff8914aa93c40f68d4a52bc8271e0c2 (127.7.111.65:36087): tablet 648d9eb802644e709f7c77b5aaf6ed2c (table test-table [id=0a5f9057080e4da8866a250467c241cb]) successfully deleted
I20260811 12:46:33.908474 11591 ts_tablet_manager.cc:1918] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:46:33.908674 11591 log.cc:1196] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/648d9eb802644e709f7c77b5aaf6ed2c
I20260811 12:46:33.909296 11591 ts_tablet_manager.cc:1939] T 648d9eb802644e709f7c77b5aaf6ed2c P e02640acffd14a7dad0476d33716e47f: Deleting consensus metadata
I20260811 12:46:33.911080 10592 catalog_manager.cc:4841] TS e02640acffd14a7dad0476d33716e47f (127.7.111.67:41823): tablet 648d9eb802644e709f7c77b5aaf6ed2c (table test-table [id=0a5f9057080e4da8866a250467c241cb]) successfully deleted
I20260811 12:46:34.097651 11561 raft_consensus.cc:482] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:46:34.097905 11561 raft_consensus.cc:504] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:34.098842 11561 leader_election.cc:288] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4955defa995f4f5f8c510f1ad24c1464 (127.7.111.66:42617), e02640acffd14a7dad0476d33716e47f (127.7.111.67:41823)
I20260811 12:46:34.099581 11135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9efd2a3b86ac4cfd99a6f7240f268c19" candidate_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4955defa995f4f5f8c510f1ad24c1464" is_pre_election: true
I20260811 12:46:34.099691 11385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9efd2a3b86ac4cfd99a6f7240f268c19" candidate_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e02640acffd14a7dad0476d33716e47f" is_pre_election: true
I20260811 12:46:34.099925 11135 raft_consensus.cc:2454] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ff8914aa93c40f68d4a52bc8271e0c2 in term 0.
I20260811 12:46:34.099998 11385 raft_consensus.cc:2454] T 9efd2a3b86ac4cfd99a6f7240f268c19 P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ff8914aa93c40f68d4a52bc8271e0c2 in term 0.
I20260811 12:46:34.100714 10762 leader_election.cc:302] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ff8914aa93c40f68d4a52bc8271e0c2, e02640acffd14a7dad0476d33716e47f; no voters:
I20260811 12:46:34.101212 11561 raft_consensus.cc:2781] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:46:34.101330 11561 raft_consensus.cc:482] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:46:34.101451 11561 raft_consensus.cc:3037] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:34.103718 11561 raft_consensus.cc:504] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:34.104729 11135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9efd2a3b86ac4cfd99a6f7240f268c19" candidate_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4955defa995f4f5f8c510f1ad24c1464"
I20260811 12:46:34.104758 11385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9efd2a3b86ac4cfd99a6f7240f268c19" candidate_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e02640acffd14a7dad0476d33716e47f"
I20260811 12:46:34.104952 11135 raft_consensus.cc:3037] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:34.104947 11385 raft_consensus.cc:3037] T 9efd2a3b86ac4cfd99a6f7240f268c19 P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:34.108019 11135 raft_consensus.cc:2454] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 4955defa995f4f5f8c510f1ad24c1464 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ff8914aa93c40f68d4a52bc8271e0c2 in term 1.
I20260811 12:46:34.108306 11561 leader_election.cc:288] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2 [CANDIDATE]: Term 1 election: Requested vote from peers 4955defa995f4f5f8c510f1ad24c1464 (127.7.111.66:42617), e02640acffd14a7dad0476d33716e47f (127.7.111.67:41823)
I20260811 12:46:34.108481 11385 raft_consensus.cc:2454] T 9efd2a3b86ac4cfd99a6f7240f268c19 P e02640acffd14a7dad0476d33716e47f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ff8914aa93c40f68d4a52bc8271e0c2 in term 1.
I20260811 12:46:34.108604 10764 leader_election.cc:302] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4955defa995f4f5f8c510f1ad24c1464, 6ff8914aa93c40f68d4a52bc8271e0c2; no voters:
I20260811 12:46:34.109045 11561 raft_consensus.cc:2781] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:46:34.109232 11561 raft_consensus.cc:686] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 1 LEADER]: Becoming Leader. State: Replica: 6ff8914aa93c40f68d4a52bc8271e0c2, State: Running, Role: LEADER
I20260811 12:46:34.109651 11561 consensus_queue.cc:237] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:34.115404 10613 catalog_manager.cc:5495] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ff8914aa93c40f68d4a52bc8271e0c2 (127.7.111.65). New cstate: current_term: 1 leader_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } health_report { overall_health: UNKNOWN } } }
I20260811 12:46:34.147725 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:34.141765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33590 (local address 127.7.111.65:36087)
0811 12:46:34.141963 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:34.141969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:34.142004 (+ 35us) server_negotiation.cc:407] Connection header received
0811 12:46:34.142232 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:34.142238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:34.142305 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:34.142374 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:46:34.142806 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:34.143653 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:34.145782 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:34.146233 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:34.146324 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:34.146966 (+ 642us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:46:34.147077 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:46:34.147143 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:34.147331 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 12:46:34.147398 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:46:34.161091 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:34.156493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:36785 (local address 127.7.111.65:36087)
0811 12:46:34.156632 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:34.156637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:34.156674 (+ 37us) server_negotiation.cc:407] Connection header received
0811 12:46:34.156850 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:34.156854 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:34.156910 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:34.156977 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:46:34.157287 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:34.158122 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:34.159543 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:34.159924 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:34.160025 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:34.160419 (+ 394us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:46:34.160542 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:46:34.160615 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:34.160741 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 12:46:34.160805 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 12:46:34.161096 11586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:34.156411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:36087 (local address 127.7.111.65:36785)
0811 12:46:34.156600 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:34.156619 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:34.156745 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:34.157000 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:34.157004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:34.157015 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:46:34.157173 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:34.157182 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:34.158290 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:34.158295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:34.159416 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:34.159429 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:34.160061 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:34.160067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:34.160242 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:34.160278 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:46:34.160632 (+ 354us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:46:34.160644 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 12:46:34.160716 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 12:46:34.160845 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 12:46:34.175328 11135 raft_consensus.cc:1261] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 4955defa995f4f5f8c510f1ad24c1464 [term 1 FOLLOWER]: Refusing update from remote peer 6ff8914aa93c40f68d4a52bc8271e0c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:46:34.175336 11385 raft_consensus.cc:1261] T 9efd2a3b86ac4cfd99a6f7240f268c19 P e02640acffd14a7dad0476d33716e47f [term 1 FOLLOWER]: Refusing update from remote peer 6ff8914aa93c40f68d4a52bc8271e0c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:46:34.175998 11508 consensus_queue.cc:1035] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:46:34.176282 11579 consensus_queue.cc:1035] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:46:34.193471 11553 mvcc.cc:205] Tried to move back new op lower bound from 7317309006536900608 to 7317309006276145152. Current Snapshot: MvccSnapshot[applied={T|T < 7317309006536900608}]
W20260811 12:46:34.210632 11550 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 12:46:35.806968 10613 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:46:35.807214 10613 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:46:35.811395 10613 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706: Sending DeleteTablet for 3 replicas of tablet 9efd2a3b86ac4cfd99a6f7240f268c19
I20260811 12:46:35.812130 10843 tablet_service.cc:1514] Processing DeleteTablet for tablet 9efd2a3b86ac4cfd99a6f7240f268c19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:35 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:33566
I20260811 12:46:35.812258 11090 tablet_service.cc:1514] Processing DeleteTablet for tablet 9efd2a3b86ac4cfd99a6f7240f268c19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:35 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:42932
I20260811 12:46:35.812232 11333 tablet_service.cc:1514] Processing DeleteTablet for tablet 9efd2a3b86ac4cfd99a6f7240f268c19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:35 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:47046
I20260811 12:46:35.824728 11641 tablet_replica.cc:330] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2: stopping tablet replica
I20260811 12:46:35.825089 11641 raft_consensus.cc:2229] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:46:35.825796 11641 raft_consensus.cc:2258] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:46:35.827121 11641 ts_tablet_manager.cc:1905] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:46:35.830534 11644 tablet_replica.cc:330] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 4955defa995f4f5f8c510f1ad24c1464: stopping tablet replica
I20260811 12:46:35.830785 11646 tablet_replica.cc:330] T 9efd2a3b86ac4cfd99a6f7240f268c19 P e02640acffd14a7dad0476d33716e47f: stopping tablet replica
I20260811 12:46:35.830791 11644 raft_consensus.cc:2229] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 4955defa995f4f5f8c510f1ad24c1464 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:46:35.831037 11646 raft_consensus.cc:2229] T 9efd2a3b86ac4cfd99a6f7240f268c19 P e02640acffd14a7dad0476d33716e47f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:46:35.831336 11646 raft_consensus.cc:2258] T 9efd2a3b86ac4cfd99a6f7240f268c19 P e02640acffd14a7dad0476d33716e47f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:46:35.831326 11644 raft_consensus.cc:2258] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 4955defa995f4f5f8c510f1ad24c1464 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:46:35.832453 11646 ts_tablet_manager.cc:1905] T 9efd2a3b86ac4cfd99a6f7240f268c19 P e02640acffd14a7dad0476d33716e47f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:46:35.832530 11644 ts_tablet_manager.cc:1905] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 4955defa995f4f5f8c510f1ad24c1464: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:46:35.835075 11641 ts_tablet_manager.cc:1918] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:46:35.835243 11641 log.cc:1196] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/9efd2a3b86ac4cfd99a6f7240f268c19
I20260811 12:46:35.835743 11641 ts_tablet_manager.cc:1939] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 6ff8914aa93c40f68d4a52bc8271e0c2: Deleting consensus metadata
I20260811 12:46:35.837167 10592 catalog_manager.cc:4841] TS 6ff8914aa93c40f68d4a52bc8271e0c2 (127.7.111.65:36087): tablet 9efd2a3b86ac4cfd99a6f7240f268c19 (table test-table [id=24b65652297948f3a5c8aac1937547bd]) successfully deleted
I20260811 12:46:35.838956 11646 ts_tablet_manager.cc:1918] T 9efd2a3b86ac4cfd99a6f7240f268c19 P e02640acffd14a7dad0476d33716e47f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:46:35.839160 11646 log.cc:1196] T 9efd2a3b86ac4cfd99a6f7240f268c19 P e02640acffd14a7dad0476d33716e47f: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/9efd2a3b86ac4cfd99a6f7240f268c19
I20260811 12:46:35.839690 11646 ts_tablet_manager.cc:1939] T 9efd2a3b86ac4cfd99a6f7240f268c19 P e02640acffd14a7dad0476d33716e47f: Deleting consensus metadata
I20260811 12:46:35.840304 11644 ts_tablet_manager.cc:1918] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 4955defa995f4f5f8c510f1ad24c1464: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:46:35.840449 11644 log.cc:1196] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 4955defa995f4f5f8c510f1ad24c1464: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/9efd2a3b86ac4cfd99a6f7240f268c19
I20260811 12:46:35.840847 10592 catalog_manager.cc:4841] TS e02640acffd14a7dad0476d33716e47f (127.7.111.67:41823): tablet 9efd2a3b86ac4cfd99a6f7240f268c19 (table test-table [id=24b65652297948f3a5c8aac1937547bd]) successfully deleted
I20260811 12:46:35.841090 11644 ts_tablet_manager.cc:1939] T 9efd2a3b86ac4cfd99a6f7240f268c19 P 4955defa995f4f5f8c510f1ad24c1464: Deleting consensus metadata
I20260811 12:46:35.842096 10595 catalog_manager.cc:4841] TS 4955defa995f4f5f8c510f1ad24c1464 (127.7.111.66:42617): tablet 9efd2a3b86ac4cfd99a6f7240f268c19 (table test-table [id=24b65652297948f3a5c8aac1937547bd]) successfully deleted
Aug 11 12:46:35 dist-test-slave-f0tn krb5kdc[10538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452395, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 12:46:35 dist-test-slave-f0tn krb5kdc[10538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452395, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:35.880173 11660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:35.862709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52548 (local address 127.7.111.126:32907)
0811 12:46:35.863318 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:35.863324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:35.863467 (+ 143us) server_negotiation.cc:407] Connection header received
0811 12:46:35.863654 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:35.863658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:35.863730 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:35.863821 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:35.865170 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:35.866039 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:35.867370 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:35.867800 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:35.867908 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:35.874846 (+ 6938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:35.874852 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:35.874855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:35.874890 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:35.877756 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:35.878350 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:35.878354 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:35.878356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:35.878476 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:35.878810 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:35.878815 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:35.878818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:35.879089 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:35.879314 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:35.879601 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 12:46:35.879823 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":281,"threads_started":1}
I20260811 12:46:35.883410 10613 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:46:35.883984 10613 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:46:35.890082 10843 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca9c13041ec4aa8a21de3df69bb886d (DEFAULT_TABLE table=test-table [id=7e1feac0fdd649b2a9a8caa433c13024]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:35.890194 11090 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca9c13041ec4aa8a21de3df69bb886d (DEFAULT_TABLE table=test-table [id=7e1feac0fdd649b2a9a8caa433c13024]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:35.890384 11333 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca9c13041ec4aa8a21de3df69bb886d (DEFAULT_TABLE table=test-table [id=7e1feac0fdd649b2a9a8caa433c13024]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:35.890534 10843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ca9c13041ec4aa8a21de3df69bb886d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:35.890641 11090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ca9c13041ec4aa8a21de3df69bb886d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:35.890771 11333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ca9c13041ec4aa8a21de3df69bb886d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:35.896878 11662 tablet_bootstrap.cc:492] T 6ca9c13041ec4aa8a21de3df69bb886d P 6ff8914aa93c40f68d4a52bc8271e0c2: Bootstrap starting.
I20260811 12:46:35.897794 11665 tablet_bootstrap.cc:492] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464: Bootstrap starting.
I20260811 12:46:35.898967 11662 tablet_bootstrap.cc:654] T 6ca9c13041ec4aa8a21de3df69bb886d P 6ff8914aa93c40f68d4a52bc8271e0c2: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:35.899761 11665 tablet_bootstrap.cc:654] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:35.901403 11662 tablet_bootstrap.cc:492] T 6ca9c13041ec4aa8a21de3df69bb886d P 6ff8914aa93c40f68d4a52bc8271e0c2: No bootstrap required, opened a new log
I20260811 12:46:35.900738 11663 tablet_bootstrap.cc:492] T 6ca9c13041ec4aa8a21de3df69bb886d P e02640acffd14a7dad0476d33716e47f: Bootstrap starting.
I20260811 12:46:35.901552 11662 ts_tablet_manager.cc:1397] T 6ca9c13041ec4aa8a21de3df69bb886d P 6ff8914aa93c40f68d4a52bc8271e0c2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 12:46:35.902081 11665 tablet_bootstrap.cc:492] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464: No bootstrap required, opened a new log
I20260811 12:46:35.902251 11665 ts_tablet_manager.cc:1397] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 12:46:35.902328 11662 raft_consensus.cc:348] T 6ca9c13041ec4aa8a21de3df69bb886d P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:35.902587 11662 raft_consensus.cc:374] T 6ca9c13041ec4aa8a21de3df69bb886d P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:35.902686 11662 raft_consensus.cc:729] T 6ca9c13041ec4aa8a21de3df69bb886d P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ff8914aa93c40f68d4a52bc8271e0c2, State: Initialized, Role: FOLLOWER
I20260811 12:46:35.902962 11662 consensus_queue.cc:260] T 6ca9c13041ec4aa8a21de3df69bb886d P 6ff8914aa93c40f68d4a52bc8271e0c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:35.903016 11665 raft_consensus.cc:348] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:35.903306 11665 raft_consensus.cc:374] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:35.903419 11665 raft_consensus.cc:729] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4955defa995f4f5f8c510f1ad24c1464, State: Initialized, Role: FOLLOWER
I20260811 12:46:35.903487 11663 tablet_bootstrap.cc:654] T 6ca9c13041ec4aa8a21de3df69bb886d P e02640acffd14a7dad0476d33716e47f: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:35.903676 11665 consensus_queue.cc:260] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:35.904347 11662 ts_tablet_manager.cc:1428] T 6ca9c13041ec4aa8a21de3df69bb886d P 6ff8914aa93c40f68d4a52bc8271e0c2: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 12:46:35.905669 11665 ts_tablet_manager.cc:1428] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 12:46:35.905819 11663 tablet_bootstrap.cc:492] T 6ca9c13041ec4aa8a21de3df69bb886d P e02640acffd14a7dad0476d33716e47f: No bootstrap required, opened a new log
I20260811 12:46:35.906003 11663 ts_tablet_manager.cc:1397] T 6ca9c13041ec4aa8a21de3df69bb886d P e02640acffd14a7dad0476d33716e47f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 12:46:35.906874 11663 raft_consensus.cc:348] T 6ca9c13041ec4aa8a21de3df69bb886d P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:35.907122 11663 raft_consensus.cc:374] T 6ca9c13041ec4aa8a21de3df69bb886d P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:35.907226 11663 raft_consensus.cc:729] T 6ca9c13041ec4aa8a21de3df69bb886d P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e02640acffd14a7dad0476d33716e47f, State: Initialized, Role: FOLLOWER
I20260811 12:46:35.907474 11663 consensus_queue.cc:260] T 6ca9c13041ec4aa8a21de3df69bb886d P e02640acffd14a7dad0476d33716e47f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:35.908301 11663 ts_tablet_manager.cc:1428] T 6ca9c13041ec4aa8a21de3df69bb886d P e02640acffd14a7dad0476d33716e47f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 12:46:35.969134 11671 raft_consensus.cc:482] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:46:35.969369 11671 raft_consensus.cc:504] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:35.971001 11671 leader_election.cc:288] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ff8914aa93c40f68d4a52bc8271e0c2 (127.7.111.65:36087), e02640acffd14a7dad0476d33716e47f (127.7.111.67:41823)
I20260811 12:46:35.978073 11676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:35.971238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:36087 (local address 127.7.111.66:38563)
0811 12:46:35.971917 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:35.971935 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:35.972076 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:35.972588 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:35.972593 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:35.972626 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:35.972996 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:35.973010 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:35.974298 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:35.974304 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:35.976159 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:35.976174 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:35.977497 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:35.977501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:35.977620 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:35.977644 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 12:46:35.977718 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 12:46:35.977808 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":381,"threads_started":1}
I20260811 12:46:35.978292 11678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:35.971316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:38563 (local address 127.7.111.65:36087)
0811 12:46:35.971995 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:35.972001 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:35.972062 (+ 61us) server_negotiation.cc:407] Connection header received
0811 12:46:35.972221 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:35.972226 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:35.972314 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:35.972415 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:35.973253 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:35.974168 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:35.976333 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:35.977300 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:35.977515 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:35.977589 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:35.977760 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 12:46:35.977902 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":359,"threads_started":1}
I20260811 12:46:35.978892 10880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca9c13041ec4aa8a21de3df69bb886d" candidate_uuid: "4955defa995f4f5f8c510f1ad24c1464" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" is_pre_election: true
I20260811 12:46:35.979182 10880 raft_consensus.cc:2454] T 6ca9c13041ec4aa8a21de3df69bb886d P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4955defa995f4f5f8c510f1ad24c1464 in term 0.
I20260811 12:46:35.979339 11681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:35.972211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:58575 (local address 127.7.111.67:41823)
0811 12:46:35.973168 (+ 957us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:35.973173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:35.973196 (+ 23us) server_negotiation.cc:407] Connection header received
0811 12:46:35.973262 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:35.973265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:35.973320 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:35.973429 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:35.974059 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:35.975162 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:35.977180 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:35.978436 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:35.978657 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:35.978731 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:35.978962 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 12:46:35.979062 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":819,"thread_start_us":655,"threads_started":1}
I20260811 12:46:35.979522 11679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:35.972036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:41823 (local address 127.7.111.66:58575)
0811 12:46:35.972773 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:35.972799 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:35.972929 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:35.973442 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:35.973446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:35.973470 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:35.973897 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:35.973907 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:35.975300 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:35.975307 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:35.977010 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:35.977023 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:35.978648 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:35.978652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:35.978815 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:35.978846 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:46:35.978945 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 12:46:35.979062 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":340,"threads_started":1}
I20260811 12:46:35.979838 11006 leader_election.cc:302] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4955defa995f4f5f8c510f1ad24c1464, 6ff8914aa93c40f68d4a52bc8271e0c2; no voters:
I20260811 12:46:35.980224 11671 raft_consensus.cc:2781] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:46:35.980366 11671 raft_consensus.cc:482] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:46:35.980504 11671 raft_consensus.cc:3037] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:35.980597 11384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca9c13041ec4aa8a21de3df69bb886d" candidate_uuid: "4955defa995f4f5f8c510f1ad24c1464" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e02640acffd14a7dad0476d33716e47f" is_pre_election: true
I20260811 12:46:35.980820 11384 raft_consensus.cc:2454] T 6ca9c13041ec4aa8a21de3df69bb886d P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4955defa995f4f5f8c510f1ad24c1464 in term 0.
I20260811 12:46:35.983035 11671 raft_consensus.cc:504] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:35.983664 11671 leader_election.cc:288] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464 [CANDIDATE]: Term 1 election: Requested vote from peers 6ff8914aa93c40f68d4a52bc8271e0c2 (127.7.111.65:36087), e02640acffd14a7dad0476d33716e47f (127.7.111.67:41823)
I20260811 12:46:35.984061 10880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca9c13041ec4aa8a21de3df69bb886d" candidate_uuid: "4955defa995f4f5f8c510f1ad24c1464" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2"
I20260811 12:46:35.984125 11384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca9c13041ec4aa8a21de3df69bb886d" candidate_uuid: "4955defa995f4f5f8c510f1ad24c1464" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e02640acffd14a7dad0476d33716e47f"
I20260811 12:46:35.984233 10880 raft_consensus.cc:3037] T 6ca9c13041ec4aa8a21de3df69bb886d P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:35.984275 11384 raft_consensus.cc:3037] T 6ca9c13041ec4aa8a21de3df69bb886d P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:35.986502 11384 raft_consensus.cc:2454] T 6ca9c13041ec4aa8a21de3df69bb886d P e02640acffd14a7dad0476d33716e47f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4955defa995f4f5f8c510f1ad24c1464 in term 1.
I20260811 12:46:35.986505 10880 raft_consensus.cc:2454] T 6ca9c13041ec4aa8a21de3df69bb886d P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4955defa995f4f5f8c510f1ad24c1464 in term 1.
I20260811 12:46:35.986843 11006 leader_election.cc:302] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4955defa995f4f5f8c510f1ad24c1464, 6ff8914aa93c40f68d4a52bc8271e0c2; no voters:
I20260811 12:46:35.987110 11671 raft_consensus.cc:2781] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:46:35.987637 11671 raft_consensus.cc:686] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464 [term 1 LEADER]: Becoming Leader. State: Replica: 4955defa995f4f5f8c510f1ad24c1464, State: Running, Role: LEADER
I20260811 12:46:35.988133 11671 consensus_queue.cc:237] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:35.991261 10613 catalog_manager.cc:5495] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4955defa995f4f5f8c510f1ad24c1464 (127.7.111.66). New cstate: current_term: 1 leader_uuid: "4955defa995f4f5f8c510f1ad24c1464" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } health_report { overall_health: UNKNOWN } } }
I20260811 12:46:36.031416 11690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:36.025417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49756 (local address 127.7.111.66:42617)
0811 12:46:36.026185 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:36.026189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:36.026210 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:46:36.026326 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:36.026330 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:36.026410 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:36.026554 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:46:36.026837 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:36.027666 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:36.028962 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:36.029532 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:36.029632 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:36.030064 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:46:36.030714 (+ 650us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:46:36.030846 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:36.030958 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 12:46:36.031036 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":485,"threads_started":1}
I20260811 12:46:36.047777 11678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:36.042292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:43795 (local address 127.7.111.65:36087)
0811 12:46:36.042495 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:36.042501 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:36.042995 (+ 494us) server_negotiation.cc:407] Connection header received
0811 12:46:36.043184 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:36.043188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:36.043259 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:36.043332 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:46:36.043834 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:36.044570 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:36.046021 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:36.046485 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:36.046598 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:36.047062 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:46:36.047185 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:46:36.047259 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:36.047396 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 12:46:36.047477 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 12:46:36.047914 11694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:36.042139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:36087 (local address 127.7.111.66:43795)
0811 12:46:36.042893 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:36.042909 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:36.043054 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:36.043365 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:36.043369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:36.043382 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:46:36.043681 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:36.043691 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:36.044700 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:36.044706 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:36.045885 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:36.045898 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:36.046657 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:36.046664 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:36.046837 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:36.046894 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:46:36.047279 (+ 385us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:46:36.047300 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 12:46:36.047367 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 12:46:36.047495 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":450,"threads_started":1}
I20260811 12:46:36.055663 10880 raft_consensus.cc:1261] T 6ca9c13041ec4aa8a21de3df69bb886d P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 1 FOLLOWER]: Refusing update from remote peer 4955defa995f4f5f8c510f1ad24c1464: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:46:36.055694 11384 raft_consensus.cc:1261] T 6ca9c13041ec4aa8a21de3df69bb886d P e02640acffd14a7dad0476d33716e47f [term 1 FOLLOWER]: Refusing update from remote peer 4955defa995f4f5f8c510f1ad24c1464: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:46:36.056334 11683 consensus_queue.cc:1035] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:46:36.056771 11671 consensus_queue.cc:1035] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:46:36.083935 11690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:36.077998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:33207 (local address 127.7.111.66:42617)
0811 12:46:36.078151 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:36.078157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:36.078973 (+ 816us) server_negotiation.cc:407] Connection header received
0811 12:46:36.079093 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:36.079097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:36.079158 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:36.079238 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:46:36.079705 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:36.080566 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:36.082498 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:36.082911 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:36.083003 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:36.083336 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:46:36.083434 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:46:36.083499 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:36.083589 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 12:46:36.083660 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 12:46:36.084002 11703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:36.078032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:42617 (local address 127.7.111.65:33207)
0811 12:46:36.078877 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:36.078898 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:36.079008 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:36.079243 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:36.079246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:36.079254 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:46:36.079574 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:36.079584 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:36.080712 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:36.080717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:36.082372 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:36.082386 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:36.083042 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:36.083046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:36.083170 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:36.083216 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:46:36.083504 (+ 288us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:46:36.083511 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 12:46:36.083569 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 12:46:36.083682 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":575,"threads_started":1}
I20260811 12:46:36.110534 10613 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:46:36.110785 10613 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:46:36.113843 10613 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706: Sending DeleteTablet for 3 replicas of tablet 6ca9c13041ec4aa8a21de3df69bb886d
I20260811 12:46:36.114739 11090 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ca9c13041ec4aa8a21de3df69bb886d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:36 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:42932
I20260811 12:46:36.115042 11644 tablet_replica.cc:330] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464: stopping tablet replica
I20260811 12:46:36.115075 11333 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ca9c13041ec4aa8a21de3df69bb886d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:36 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:47046
I20260811 12:46:36.115031 10843 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ca9c13041ec4aa8a21de3df69bb886d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:36 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:33566
I20260811 12:46:36.115252 11644 raft_consensus.cc:2229] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:46:36.115336 11646 tablet_replica.cc:330] T 6ca9c13041ec4aa8a21de3df69bb886d P e02640acffd14a7dad0476d33716e47f: stopping tablet replica
I20260811 12:46:36.115523 11646 raft_consensus.cc:2229] T 6ca9c13041ec4aa8a21de3df69bb886d P e02640acffd14a7dad0476d33716e47f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:46:36.115742 11646 raft_consensus.cc:2258] T 6ca9c13041ec4aa8a21de3df69bb886d P e02640acffd14a7dad0476d33716e47f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:46:36.115815 11644 raft_consensus.cc:2258] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:46:36.115872 11641 tablet_replica.cc:330] T 6ca9c13041ec4aa8a21de3df69bb886d P 6ff8914aa93c40f68d4a52bc8271e0c2: stopping tablet replica
I20260811 12:46:36.116144 11641 raft_consensus.cc:2229] T 6ca9c13041ec4aa8a21de3df69bb886d P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:46:36.116394 11641 raft_consensus.cc:2258] T 6ca9c13041ec4aa8a21de3df69bb886d P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:46:36.116972 11646 ts_tablet_manager.cc:1905] T 6ca9c13041ec4aa8a21de3df69bb886d P e02640acffd14a7dad0476d33716e47f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:46:36.116983 11644 ts_tablet_manager.cc:1905] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:46:36.118500 11641 ts_tablet_manager.cc:1905] T 6ca9c13041ec4aa8a21de3df69bb886d P 6ff8914aa93c40f68d4a52bc8271e0c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:46:36.123594 11646 ts_tablet_manager.cc:1918] T 6ca9c13041ec4aa8a21de3df69bb886d P e02640acffd14a7dad0476d33716e47f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:46:36.123749 11646 log.cc:1196] T 6ca9c13041ec4aa8a21de3df69bb886d P e02640acffd14a7dad0476d33716e47f: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/6ca9c13041ec4aa8a21de3df69bb886d
I20260811 12:46:36.124074 11644 ts_tablet_manager.cc:1918] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:46:36.124159 11646 ts_tablet_manager.cc:1939] T 6ca9c13041ec4aa8a21de3df69bb886d P e02640acffd14a7dad0476d33716e47f: Deleting consensus metadata
I20260811 12:46:36.124253 11644 log.cc:1196] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/6ca9c13041ec4aa8a21de3df69bb886d
I20260811 12:46:36.124647 11644 ts_tablet_manager.cc:1939] T 6ca9c13041ec4aa8a21de3df69bb886d P 4955defa995f4f5f8c510f1ad24c1464: Deleting consensus metadata
I20260811 12:46:36.124379 10613 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 12:46:36.125269 10592 catalog_manager.cc:4841] TS e02640acffd14a7dad0476d33716e47f (127.7.111.67:41823): tablet 6ca9c13041ec4aa8a21de3df69bb886d (table test-table [id=7e1feac0fdd649b2a9a8caa433c13024]) successfully deleted
I20260811 12:46:36.125698 10595 catalog_manager.cc:4841] TS 4955defa995f4f5f8c510f1ad24c1464 (127.7.111.66:42617): tablet 6ca9c13041ec4aa8a21de3df69bb886d (table test-table [id=7e1feac0fdd649b2a9a8caa433c13024]) successfully deleted
I20260811 12:46:36.127871 11641 ts_tablet_manager.cc:1918] T 6ca9c13041ec4aa8a21de3df69bb886d P 6ff8914aa93c40f68d4a52bc8271e0c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:46:36.128096 11641 log.cc:1196] T 6ca9c13041ec4aa8a21de3df69bb886d P 6ff8914aa93c40f68d4a52bc8271e0c2: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/6ca9c13041ec4aa8a21de3df69bb886d
W20260811 12:46:36.128149 10613 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:46:36.128554 11641 ts_tablet_manager.cc:1939] T 6ca9c13041ec4aa8a21de3df69bb886d P 6ff8914aa93c40f68d4a52bc8271e0c2: Deleting consensus metadata
I20260811 12:46:36.129707 10592 catalog_manager.cc:4841] TS 6ff8914aa93c40f68d4a52bc8271e0c2 (127.7.111.65:36087): tablet 6ca9c13041ec4aa8a21de3df69bb886d (table test-table [id=7e1feac0fdd649b2a9a8caa433c13024]) successfully deleted
I20260811 12:46:36.134742 10843 tablet_service.cc:1467] Processing CreateTablet for tablet 38bf7dead3b640578523a849b125588a (DEFAULT_TABLE table=test-table [id=f97683216810449b9c36bf704dc4b3e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:36.134804 11090 tablet_service.cc:1467] Processing CreateTablet for tablet 38bf7dead3b640578523a849b125588a (DEFAULT_TABLE table=test-table [id=f97683216810449b9c36bf704dc4b3e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:36.135238 11090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38bf7dead3b640578523a849b125588a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:36.135370 10843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38bf7dead3b640578523a849b125588a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:36.141045 11665 tablet_bootstrap.cc:492] T 38bf7dead3b640578523a849b125588a P 4955defa995f4f5f8c510f1ad24c1464: Bootstrap starting.
I20260811 12:46:36.143193 11665 tablet_bootstrap.cc:654] T 38bf7dead3b640578523a849b125588a P 4955defa995f4f5f8c510f1ad24c1464: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:36.145102 11333 tablet_service.cc:1467] Processing CreateTablet for tablet 38bf7dead3b640578523a849b125588a (DEFAULT_TABLE table=test-table [id=f97683216810449b9c36bf704dc4b3e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:36.145602 11333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38bf7dead3b640578523a849b125588a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:36.145882 11665 tablet_bootstrap.cc:492] T 38bf7dead3b640578523a849b125588a P 4955defa995f4f5f8c510f1ad24c1464: No bootstrap required, opened a new log
I20260811 12:46:36.146170 11665 ts_tablet_manager.cc:1397] T 38bf7dead3b640578523a849b125588a P 4955defa995f4f5f8c510f1ad24c1464: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 12:46:36.147284 11665 raft_consensus.cc:348] T 38bf7dead3b640578523a849b125588a P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:36.147658 11665 raft_consensus.cc:374] T 38bf7dead3b640578523a849b125588a P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:36.147763 11665 raft_consensus.cc:729] T 38bf7dead3b640578523a849b125588a P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4955defa995f4f5f8c510f1ad24c1464, State: Initialized, Role: FOLLOWER
I20260811 12:46:36.148054 11665 consensus_queue.cc:260] T 38bf7dead3b640578523a849b125588a P 4955defa995f4f5f8c510f1ad24c1464 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:36.148885 11665 ts_tablet_manager.cc:1428] T 38bf7dead3b640578523a849b125588a P 4955defa995f4f5f8c510f1ad24c1464: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 12:46:36.150311 11662 tablet_bootstrap.cc:492] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2: Bootstrap starting.
I20260811 12:46:36.151885 11662 tablet_bootstrap.cc:654] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:36.154157 11663 tablet_bootstrap.cc:492] T 38bf7dead3b640578523a849b125588a P e02640acffd14a7dad0476d33716e47f: Bootstrap starting.
I20260811 12:46:36.155607 11663 tablet_bootstrap.cc:654] T 38bf7dead3b640578523a849b125588a P e02640acffd14a7dad0476d33716e47f: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:36.162700 11662 tablet_bootstrap.cc:492] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2: No bootstrap required, opened a new log
I20260811 12:46:36.162926 11662 ts_tablet_manager.cc:1397] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20260811 12:46:36.166829 11662 raft_consensus.cc:348] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:36.167264 11662 raft_consensus.cc:374] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:36.167424 11662 raft_consensus.cc:729] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ff8914aa93c40f68d4a52bc8271e0c2, State: Initialized, Role: FOLLOWER
I20260811 12:46:36.167804 11662 consensus_queue.cc:260] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:36.168754 11662 ts_tablet_manager.cc:1428] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260811 12:46:36.170403 11663 tablet_bootstrap.cc:492] T 38bf7dead3b640578523a849b125588a P e02640acffd14a7dad0476d33716e47f: No bootstrap required, opened a new log
I20260811 12:46:36.170579 11663 ts_tablet_manager.cc:1397] T 38bf7dead3b640578523a849b125588a P e02640acffd14a7dad0476d33716e47f: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.011s
I20260811 12:46:36.171334 11663 raft_consensus.cc:348] T 38bf7dead3b640578523a849b125588a P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:36.171577 11663 raft_consensus.cc:374] T 38bf7dead3b640578523a849b125588a P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:36.171654 11663 raft_consensus.cc:729] T 38bf7dead3b640578523a849b125588a P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e02640acffd14a7dad0476d33716e47f, State: Initialized, Role: FOLLOWER
I20260811 12:46:36.171929 11663 consensus_queue.cc:260] T 38bf7dead3b640578523a849b125588a P e02640acffd14a7dad0476d33716e47f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:36.172474 11663 ts_tablet_manager.cc:1428] T 38bf7dead3b640578523a849b125588a P e02640acffd14a7dad0476d33716e47f: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 12:46:36.200933 11615 raft_consensus.cc:482] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:46:36.201105 11615 raft_consensus.cc:504] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:36.202077 11615 leader_election.cc:288] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4955defa995f4f5f8c510f1ad24c1464 (127.7.111.66:42617), e02640acffd14a7dad0476d33716e47f (127.7.111.67:41823)
I20260811 12:46:36.202612 11135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38bf7dead3b640578523a849b125588a" candidate_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4955defa995f4f5f8c510f1ad24c1464" is_pre_election: true
I20260811 12:46:36.202807 11385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38bf7dead3b640578523a849b125588a" candidate_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e02640acffd14a7dad0476d33716e47f" is_pre_election: true
I20260811 12:46:36.202993 11135 raft_consensus.cc:2454] T 38bf7dead3b640578523a849b125588a P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ff8914aa93c40f68d4a52bc8271e0c2 in term 0.
I20260811 12:46:36.203100 11385 raft_consensus.cc:2454] T 38bf7dead3b640578523a849b125588a P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ff8914aa93c40f68d4a52bc8271e0c2 in term 0.
I20260811 12:46:36.203444 10764 leader_election.cc:302] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4955defa995f4f5f8c510f1ad24c1464, 6ff8914aa93c40f68d4a52bc8271e0c2; no voters:
I20260811 12:46:36.203735 11615 raft_consensus.cc:2781] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:46:36.203853 11615 raft_consensus.cc:482] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:46:36.203922 11615 raft_consensus.cc:3037] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:36.205948 11615 raft_consensus.cc:504] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:36.206494 11615 leader_election.cc:288] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2 [CANDIDATE]: Term 1 election: Requested vote from peers 4955defa995f4f5f8c510f1ad24c1464 (127.7.111.66:42617), e02640acffd14a7dad0476d33716e47f (127.7.111.67:41823)
I20260811 12:46:36.206930 11135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38bf7dead3b640578523a849b125588a" candidate_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4955defa995f4f5f8c510f1ad24c1464"
I20260811 12:46:36.207176 11135 raft_consensus.cc:3037] T 38bf7dead3b640578523a849b125588a P 4955defa995f4f5f8c510f1ad24c1464 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:36.207489 11385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38bf7dead3b640578523a849b125588a" candidate_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e02640acffd14a7dad0476d33716e47f"
I20260811 12:46:36.207759 11385 raft_consensus.cc:3037] T 38bf7dead3b640578523a849b125588a P e02640acffd14a7dad0476d33716e47f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:36.210348 11135 raft_consensus.cc:2454] T 38bf7dead3b640578523a849b125588a P 4955defa995f4f5f8c510f1ad24c1464 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ff8914aa93c40f68d4a52bc8271e0c2 in term 1.
I20260811 12:46:36.210762 10764 leader_election.cc:302] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4955defa995f4f5f8c510f1ad24c1464, 6ff8914aa93c40f68d4a52bc8271e0c2; no voters:
I20260811 12:46:36.211062 11615 raft_consensus.cc:2781] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:46:36.211236 11615 raft_consensus.cc:686] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 1 LEADER]: Becoming Leader. State: Replica: 6ff8914aa93c40f68d4a52bc8271e0c2, State: Running, Role: LEADER
I20260811 12:46:36.211519 11615 consensus_queue.cc:237] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } }
I20260811 12:46:36.211866 11385 raft_consensus.cc:2454] T 38bf7dead3b640578523a849b125588a P e02640acffd14a7dad0476d33716e47f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ff8914aa93c40f68d4a52bc8271e0c2 in term 1.
I20260811 12:46:36.215176 10611 catalog_manager.cc:5495] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ff8914aa93c40f68d4a52bc8271e0c2 (127.7.111.65). New cstate: current_term: 1 leader_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff8914aa93c40f68d4a52bc8271e0c2" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 } health_report { overall_health: UNKNOWN } } }
I20260811 12:46:36.248421 11678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:36.242574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47124 (local address 127.7.111.65:36087)
0811 12:46:36.242766 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:36.242772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:36.242973 (+ 201us) server_negotiation.cc:407] Connection header received
0811 12:46:36.243170 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:36.243174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:36.243239 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:36.243309 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:46:36.243693 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:36.244650 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:36.246662 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:36.247076 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:36.247165 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:36.247784 (+ 619us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:46:36.247883 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:46:36.247945 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:36.248088 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 12:46:36.248160 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:46:36.258133 11385 raft_consensus.cc:1261] T 38bf7dead3b640578523a849b125588a P e02640acffd14a7dad0476d33716e47f [term 1 FOLLOWER]: Refusing update from remote peer 6ff8914aa93c40f68d4a52bc8271e0c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:46:36.258723 11135 raft_consensus.cc:1261] T 38bf7dead3b640578523a849b125588a P 4955defa995f4f5f8c510f1ad24c1464 [term 1 FOLLOWER]: Refusing update from remote peer 6ff8914aa93c40f68d4a52bc8271e0c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:46:36.258772 11538 consensus_queue.cc:1035] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e02640acffd14a7dad0476d33716e47f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:46:36.259330 11538 consensus_queue.cc:1035] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4955defa995f4f5f8c510f1ad24c1464" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 12:46:36.284200 11613 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 12:46:37.872047 10613 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:46:37.872296 10613 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:46:37.876157 10613 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c918b7949a64e909d122c6801244706: Sending DeleteTablet for 3 replicas of tablet 38bf7dead3b640578523a849b125588a
I20260811 12:46:37.876814 10843 tablet_service.cc:1514] Processing DeleteTablet for tablet 38bf7dead3b640578523a849b125588a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:37 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:33566
I20260811 12:46:37.876924 11333 tablet_service.cc:1514] Processing DeleteTablet for tablet 38bf7dead3b640578523a849b125588a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:37 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:47046
I20260811 12:46:37.876953 11090 tablet_service.cc:1514] Processing DeleteTablet for tablet 38bf7dead3b640578523a849b125588a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:37 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:42932
I20260811 12:46:37.877738 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 10736
I20260811 12:46:37.889982 11764 tablet_replica.cc:330] T 38bf7dead3b640578523a849b125588a P 4955defa995f4f5f8c510f1ad24c1464: stopping tablet replica
I20260811 12:46:37.890216 11764 raft_consensus.cc:2229] T 38bf7dead3b640578523a849b125588a P 4955defa995f4f5f8c510f1ad24c1464 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:46:37.890466 11760 tablet_replica.cc:330] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2: stopping tablet replica
I20260811 12:46:37.890489 11764 raft_consensus.cc:2258] T 38bf7dead3b640578523a849b125588a P 4955defa995f4f5f8c510f1ad24c1464 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:46:37.890681 11760 raft_consensus.cc:2229] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:46:37.891079 11760 raft_consensus.cc:2258] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:46:37.891575 11764 ts_tablet_manager.cc:1905] T 38bf7dead3b640578523a849b125588a P 4955defa995f4f5f8c510f1ad24c1464: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:46:37.892035 11760 ts_tablet_manager.cc:1905] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:46:37.896664 11762 tablet_replica.cc:330] T 38bf7dead3b640578523a849b125588a P e02640acffd14a7dad0476d33716e47f: stopping tablet replica
I20260811 12:46:37.896993 11762 raft_consensus.cc:2229] T 38bf7dead3b640578523a849b125588a P e02640acffd14a7dad0476d33716e47f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:46:37.897320 11762 raft_consensus.cc:2258] T 38bf7dead3b640578523a849b125588a P e02640acffd14a7dad0476d33716e47f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:46:37.898376 11762 ts_tablet_manager.cc:1905] T 38bf7dead3b640578523a849b125588a P e02640acffd14a7dad0476d33716e47f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:46:37.898662 11760 ts_tablet_manager.cc:1918] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:46:37.898816 11760 log.cc:1196] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/38bf7dead3b640578523a849b125588a
I20260811 12:46:37.899011 11764 ts_tablet_manager.cc:1918] T 38bf7dead3b640578523a849b125588a P 4955defa995f4f5f8c510f1ad24c1464: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:46:37.899159 11764 log.cc:1196] T 38bf7dead3b640578523a849b125588a P 4955defa995f4f5f8c510f1ad24c1464: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/38bf7dead3b640578523a849b125588a
I20260811 12:46:37.899240 11760 ts_tablet_manager.cc:1939] T 38bf7dead3b640578523a849b125588a P 6ff8914aa93c40f68d4a52bc8271e0c2: Deleting consensus metadata
I20260811 12:46:37.899600 11764 ts_tablet_manager.cc:1939] T 38bf7dead3b640578523a849b125588a P 4955defa995f4f5f8c510f1ad24c1464: Deleting consensus metadata
Aug 11 12:46:37 dist-test-slave-f0tn krb5kdc[10538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452395, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.111.65@KRBTEST.COM
I20260811 12:46:37.900442 10592 catalog_manager.cc:4841] TS 6ff8914aa93c40f68d4a52bc8271e0c2 (127.7.111.65:36087): tablet 38bf7dead3b640578523a849b125588a (table test-table [id=f97683216810449b9c36bf704dc4b3e0]) successfully deleted
I20260811 12:46:37.900714 10595 catalog_manager.cc:4841] TS 4955defa995f4f5f8c510f1ad24c1464 (127.7.111.66:42617): tablet 38bf7dead3b640578523a849b125588a (table test-table [id=f97683216810449b9c36bf704dc4b3e0]) successfully deleted
I20260811 12:46:37.904134 11762 ts_tablet_manager.cc:1918] T 38bf7dead3b640578523a849b125588a P e02640acffd14a7dad0476d33716e47f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:46:37.904264 11762 log.cc:1196] T 38bf7dead3b640578523a849b125588a P e02640acffd14a7dad0476d33716e47f: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/38bf7dead3b640578523a849b125588a
I20260811 12:46:37.904603 11762 ts_tablet_manager.cc:1939] T 38bf7dead3b640578523a849b125588a P e02640acffd14a7dad0476d33716e47f: Deleting consensus metadata
I20260811 12:46:37.905679 10592 catalog_manager.cc:4841] TS e02640acffd14a7dad0476d33716e47f (127.7.111.67:41823): tablet 38bf7dead3b640578523a849b125588a (table test-table [id=f97683216810449b9c36bf704dc4b3e0]) successfully deleted
I20260811 12:46:37.907359 11766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:37.879967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47134 (local address 127.7.111.65:36087)
0811 12:46:37.890638 (+ 10671us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:37.890643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:37.890674 (+ 31us) server_negotiation.cc:407] Connection header received
0811 12:46:37.890739 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:37.890744 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:37.890823 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:37.890928 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:37.892715 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:37.893877 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:37.895592 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:37.896295 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:37.896417 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:37.901453 (+ 5036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:37.901473 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:37.901496 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:37.901541 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:37.904563 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:37.905316 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:37.905324 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:37.905326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:37.905499 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:37.906100 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:37.906104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:37.906106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:37.906463 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:37.906683 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:37.906922 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 12:46:37.907090 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10493,"thread_start_us":10255,"threads_started":1}
W20260811 12:46:37.907975 10964 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47134
W20260811 12:46:37.908543 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 10736: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:46:37.908741 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 10736
I20260811 12:46:37.948707 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 10977
Aug 11 12:46:37 dist-test-slave-f0tn krb5kdc[10538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452395, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.111.66@KRBTEST.COM
I20260811 12:46:37.963006 11769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:37.949511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49772 (local address 127.7.111.66:42617)
0811 12:46:37.949982 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:37.949987 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:37.950003 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:46:37.950058 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:37.950061 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:37.950120 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:37.950226 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:37.951386 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:37.952117 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:37.953270 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:37.953737 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:37.953830 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:37.957789 (+ 3959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:37.957802 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:37.957819 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:37.957855 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:37.960560 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:37.961116 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:37.961123 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:37.961125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:37.961256 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:37.961693 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:37.961697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:37.961700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:37.962066 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:37.962305 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:37.962500 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 12:46:37.962663 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":242,"threads_started":1}
W20260811 12:46:37.963672 11204 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49772
W20260811 12:46:37.964278 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 10977: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:46:37.964418 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 10977
I20260811 12:46:37.998283 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 11226
Aug 11 12:46:38 dist-test-slave-f0tn krb5kdc[10538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452395, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.111.67@KRBTEST.COM
I20260811 12:46:38.012542 11770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:37.999125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56990 (local address 127.7.111.67:41823)
0811 12:46:37.999668 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:37.999672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:37.999689 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:46:37.999753 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:37.999757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:37.999831 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:37.999920 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:38.001142 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:38.001929 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:38.003115 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:38.003480 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:38.003566 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:38.007618 (+ 4052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:38.007635 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:38.007653 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:38.007692 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:38.010308 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:38.010890 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:38.010896 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:38.010898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:38.011017 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:38.011299 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:38.011302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:38.011304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:38.011642 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:38.011875 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:38.012113 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 12:46:38.012283 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":291,"threads_started":1}
W20260811 12:46:38.013262 11456 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56990
W20260811 12:46:38.013870 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 11226: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:46:38.014024 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 11226
I20260811 12:46:38.049777 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 10564
I20260811 12:46:38.050949 10683 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:38.168862 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 10564
2026-08-11T12:46:38Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8539 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:46:38 dist-test-slave-f0tn krb5kdc[11778](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:46:38 dist-test-slave-f0tn krb5kdc[11778](info): set up 2 sockets
Aug 11 12:46:38 dist-test-slave-f0tn krb5kdc[11778](info): commencing operation
krb5kdc: starting...
W20260811 12:46:39.398324 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:46:39 dist-test-slave-f0tn krb5kdc[11778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:46:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:46:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:46:39.598449 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:37519
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:36183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:37519
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:39.921635 11796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:39.921968 11796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:39.922170 11796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:39.929332 11796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:46:39.929453 11796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:39.929538 11796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:39.929628 11796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:46:39.929692 11796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:46:39.939435 11796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:36183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:37519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:37519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:39.940794 11796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:39.942292 11796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:39.954407 11805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:39.954394 11806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:39.954394 11808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:39.955173 11796 server_base.cc:1034] running on GCE node
I20260811 12:46:39.955946 11796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:39.957371 11796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:39.958606 11796 hybrid_clock.cc:648] HybridClock initialized: now 1786452399958555 us; error 40 us; skew 500 ppm
I20260811 12:46:39.958953 11796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:39 dist-test-slave-f0tn krb5kdc[11778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452399, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:39.962073 11796 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:46:39.963531 11796 webserver.cc:460] Webserver started at http://127.7.111.126:34407/ using document root <none> and password file <none>
I20260811 12:46:39.964249 11796 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:39.964372 11796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:39.964699 11796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:39.968039 11796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "c1b0a0c780c34fdaa87e8bf9e979268d"
format_stamp: "Formatted at 2026-08-11 12:46:39 on dist-test-slave-f0tn"
server_key: "0cdefe9ee62936846dbac140a80b7b49"
server_key_iv: "a2fcda05e0a6b9bc690f6949655951cc"
server_key_version: "encryptionkey@0"
I20260811 12:46:39.968866 11796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "c1b0a0c780c34fdaa87e8bf9e979268d"
format_stamp: "Formatted at 2026-08-11 12:46:39 on dist-test-slave-f0tn"
server_key: "0cdefe9ee62936846dbac140a80b7b49"
server_key_iv: "a2fcda05e0a6b9bc690f6949655951cc"
server_key_version: "encryptionkey@0"
I20260811 12:46:39.974388 11796 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 12:46:39.978237 11823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:39.979671 11796 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 12:46:39.979863 11796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "c1b0a0c780c34fdaa87e8bf9e979268d"
format_stamp: "Formatted at 2026-08-11 12:46:39 on dist-test-slave-f0tn"
server_key: "0cdefe9ee62936846dbac140a80b7b49"
server_key_iv: "a2fcda05e0a6b9bc690f6949655951cc"
server_key_version: "encryptionkey@0"
I20260811 12:46:39.980082 11796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:40.007143 11796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:40.011562 11796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:40.011869 11796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:40.031178 11796 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:37519
I20260811 12:46:40.031179 11916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:37519 every 8 connection(s)
I20260811 12:46:40.032728 11796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:46:40.036528 11918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:40.036751 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 11796
I20260811 12:46:40.036975 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:46:40.037735 7613 external_mini_cluster.cc:1428] Setting key 26f4d4b4cc031cae4790eb6a82215163
I20260811 12:46:40.050896 11918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1b0a0c780c34fdaa87e8bf9e979268d: Bootstrap starting.
Aug 11 12:46:40 dist-test-slave-f0tn krb5kdc[11778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:40.054257 11918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1b0a0c780c34fdaa87e8bf9e979268d: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:40.055636 11918 log.cc:824] T 00000000000000000000000000000000 P c1b0a0c780c34fdaa87e8bf9e979268d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:46:40.058949 11918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1b0a0c780c34fdaa87e8bf9e979268d: No bootstrap required, opened a new log
I20260811 12:46:40.062153 11922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:40.039884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43764 (local address 127.7.111.126:37519)
0811 12:46:40.040706 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:40.040723 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:40.040786 (+ 63us) server_negotiation.cc:407] Connection header received
0811 12:46:40.042956 (+ 2170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:40.042992 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:40.043591 (+ 599us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:40.044066 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:40.045562 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:40.047508 (+ 1946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:40.048891 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:40.049940 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:40.050114 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:40.054150 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:40.054161 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:40.054181 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:40.054238 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:40.057790 (+ 3552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:40.058417 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:40.058425 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:40.058433 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:40.058641 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:40.058971 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:40.058976 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:40.058979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:40.059672 (+ 693us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:40.059972 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:40.060324 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 12:46:40.060745 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":332,"threads_started":1}
I20260811 12:46:40.067478 11918 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1b0a0c780c34fdaa87e8bf9e979268d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b0a0c780c34fdaa87e8bf9e979268d" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37519 } }
I20260811 12:46:40.067803 11918 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1b0a0c780c34fdaa87e8bf9e979268d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:40.067874 11918 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1b0a0c780c34fdaa87e8bf9e979268d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b0a0c780c34fdaa87e8bf9e979268d, State: Initialized, Role: FOLLOWER
I20260811 12:46:40.068380 11918 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1b0a0c780c34fdaa87e8bf9e979268d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b0a0c780c34fdaa87e8bf9e979268d" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37519 } }
I20260811 12:46:40.068670 11918 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1b0a0c780c34fdaa87e8bf9e979268d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:46:40.068820 11918 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1b0a0c780c34fdaa87e8bf9e979268d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:46:40.068996 11918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1b0a0c780c34fdaa87e8bf9e979268d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:40.071164 11918 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1b0a0c780c34fdaa87e8bf9e979268d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b0a0c780c34fdaa87e8bf9e979268d" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37519 } }
I20260811 12:46:40.071624 11918 leader_election.cc:302] T 00000000000000000000000000000000 P c1b0a0c780c34fdaa87e8bf9e979268d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1b0a0c780c34fdaa87e8bf9e979268d; no voters:
I20260811 12:46:40.072206 11918 leader_election.cc:288] T 00000000000000000000000000000000 P c1b0a0c780c34fdaa87e8bf9e979268d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:46:40.072335 11925 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1b0a0c780c34fdaa87e8bf9e979268d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:46:40.073025 11925 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1b0a0c780c34fdaa87e8bf9e979268d [term 1 LEADER]: Becoming Leader. State: Replica: c1b0a0c780c34fdaa87e8bf9e979268d, State: Running, Role: LEADER
I20260811 12:46:40.073599 11925 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1b0a0c780c34fdaa87e8bf9e979268d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b0a0c780c34fdaa87e8bf9e979268d" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37519 } }
I20260811 12:46:40.074080 11918 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1b0a0c780c34fdaa87e8bf9e979268d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:46:40.076179 11928 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1b0a0c780c34fdaa87e8bf9e979268d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1b0a0c780c34fdaa87e8bf9e979268d. Latest consensus state: current_term: 1 leader_uuid: "c1b0a0c780c34fdaa87e8bf9e979268d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b0a0c780c34fdaa87e8bf9e979268d" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37519 } } }
I20260811 12:46:40.076550 11928 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1b0a0c780c34fdaa87e8bf9e979268d [sys.catalog]: This master's current role is: LEADER
I20260811 12:46:40.076839 11927 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1b0a0c780c34fdaa87e8bf9e979268d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1b0a0c780c34fdaa87e8bf9e979268d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b0a0c780c34fdaa87e8bf9e979268d" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37519 } } }
I20260811 12:46:40.077152 11927 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1b0a0c780c34fdaa87e8bf9e979268d [sys.catalog]: This master's current role is: LEADER
I20260811 12:46:40.079080 11933 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:46:40.084376 11933 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:46:40.092517 11933 catalog_manager.cc:1347] Generated new cluster ID: 28d4458aa03b466882499cf5ea944a3c
I20260811 12:46:40.092633 11933 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:46:40.100847 11933 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:46:40.102244 11933 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:46:40.110142 11933 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1b0a0c780c34fdaa87e8bf9e979268d: Generated new TSK 0
I20260811 12:46:40.111191 11933 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.111.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
I20260811 12:46:40.170925 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:37519
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:36183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:40.500901 11956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:40.501184 11956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:40.501281 11956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:40.508440 11956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:40.508544 11956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:40.508661 11956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:46:40.519771 11956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:36183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:37519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:40.521173 11956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:40.522513 11956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:40.534986 11965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:40.535625 11969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:40.535707 11966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:40.536209 11956 server_base.cc:1034] running on GCE node
I20260811 12:46:40.536888 11956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:40.537813 11956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:40.539013 11956 hybrid_clock.cc:648] HybridClock initialized: now 1786452400538961 us; error 45 us; skew 500 ppm
I20260811 12:46:40.539304 11956 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:40 dist-test-slave-f0tn krb5kdc[11778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452400, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:40.542325 11956 init.cc:377] Logged in from keytab as kudu/127.7.111.65@KRBTEST.COM (short username kudu)
I20260811 12:46:40.543672 11956 webserver.cc:460] Webserver started at http://127.7.111.65:37527/ using document root <none> and password file <none>
I20260811 12:46:40.544294 11956 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:40.544396 11956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:40.544657 11956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:40.547480 11956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "32b54b5b2c114654b46b34a3fc9149ae"
format_stamp: "Formatted at 2026-08-11 12:46:40 on dist-test-slave-f0tn"
server_key: "306cff6b735071f2f2a686a6636a65e7"
server_key_iv: "c7b66450d761fbdaa592da5fdfc0c723"
server_key_version: "encryptionkey@0"
I20260811 12:46:40.548208 11956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "32b54b5b2c114654b46b34a3fc9149ae"
format_stamp: "Formatted at 2026-08-11 12:46:40 on dist-test-slave-f0tn"
server_key: "306cff6b735071f2f2a686a6636a65e7"
server_key_iv: "c7b66450d761fbdaa592da5fdfc0c723"
server_key_version: "encryptionkey@0"
I20260811 12:46:40.553210 11956 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:46:40.556557 11982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:40.557936 11956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:46:40.558066 11956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "32b54b5b2c114654b46b34a3fc9149ae"
format_stamp: "Formatted at 2026-08-11 12:46:40 on dist-test-slave-f0tn"
server_key: "306cff6b735071f2f2a686a6636a65e7"
server_key_iv: "c7b66450d761fbdaa592da5fdfc0c723"
server_key_version: "encryptionkey@0"
I20260811 12:46:40.558230 11956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:40.571470 11956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:40.575552 11956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:40.575812 11956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:40.576767 11956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:46:40.578162 11956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:40.578233 11956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:40.578331 11956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:40.578390 11956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:40.609351 11956 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:39473
I20260811 12:46:40.609490 12199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:39473 every 8 connection(s)
I20260811 12:46:40.610803 11956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:46:40.612530 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 11956
I20260811 12:46:40.612762 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:46:40.613507 7613 external_mini_cluster.cc:1428] Setting key 1a46d541597a5bd8d88cac8c49404fcd
Aug 11 12:46:40 dist-test-slave-f0tn krb5kdc[11778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452400, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:40.632896 12205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:40.613586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:57065 (local address 127.7.111.126:37519)
0811 12:46:40.614075 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:40.614081 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:40.615196 (+ 1115us) server_negotiation.cc:407] Connection header received
0811 12:46:40.617645 (+ 2449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:40.617649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:40.617731 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:40.617822 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:40.619750 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:40.620659 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:40.622797 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:40.623311 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:40.623418 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:40.627236 (+ 3818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:40.627241 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:40.627245 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:40.627279 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:40.629708 (+ 2429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:40.630588 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:40.630591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:40.630594 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:40.630715 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:40.631134 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:40.631137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:40.631139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:40.631393 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:40.631499 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:40.632448 (+ 949us) server_negotiation.cc:299] Negotiation successful
0811 12:46:40.632624 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":222,"threads_started":1}
I20260811 12:46:40.634344 12208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:40.614179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:37519 (local address 127.7.111.65:57065)
0811 12:46:40.615031 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:40.615080 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:40.617308 (+ 2228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:40.618087 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:40.618097 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:40.618796 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:40.619571 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:40.619595 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:40.620836 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:40.620841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:40.622669 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:40.622685 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:40.623532 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:40.623538 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:40.623805 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:40.624552 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:40.624576 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:40.626985 (+ 2409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:40.629844 (+ 2859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:40.629852 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:40.629870 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:40.630488 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:40.630807 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:40.630811 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:40.630814 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:40.631044 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:40.631510 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:40.631517 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:40.631965 (+ 448us) client_negotiation.cc:769] Sending connection context
0811 12:46:40.632284 (+ 319us) client_negotiation.cc:240] Negotiation successful
0811 12:46:40.632843 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":346,"threads_started":1}
I20260811 12:46:40.635903 12202 heartbeater.cc:347] Connected to a master server at 127.7.111.126:37519
I20260811 12:46:40.636243 12202 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.111.66@KRBTEST.COM; defaulting to no policy
I20260811 12:46:40.636994 12202 heartbeater.cc:511] Master 127.7.111.126:37519 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.66@KRBTEST.COM" created.
I20260811 12:46:40.639319 11853 ts_manager.cc:194] Registered new tserver with Master: 32b54b5b2c114654b46b34a3fc9149ae (127.7.111.65:39473)
I20260811 12:46:40.640815 11853 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:57065
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
I20260811 12:46:40.665000 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:37519
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:36183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:40.987455 12213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:40.987732 12213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:40.987865 12213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:40.994956 12213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:40.995082 12213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:40.995234 12213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:46:41.004716 12213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:36183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:37519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:41.006022 12213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:41.007325 12213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:41.020195 12225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:41.020222 12226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:41.020323 12230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:41.020995 12213 server_base.cc:1034] running on GCE node
I20260811 12:46:41.021624 12213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:41.022552 12213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:41.023767 12213 hybrid_clock.cc:648] HybridClock initialized: now 1786452401023720 us; error 34 us; skew 500 ppm
I20260811 12:46:41.024075 12213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:41 dist-test-slave-f0tn krb5kdc[11778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452401, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:41.030795 12213 init.cc:377] Logged in from keytab as kudu/127.7.111.66@KRBTEST.COM (short username kudu)
I20260811 12:46:41.034863 12213 webserver.cc:460] Webserver started at http://127.7.111.66:33145/ using document root <none> and password file <none>
I20260811 12:46:41.035574 12213 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:41.035683 12213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:41.035984 12213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:41.038918 12213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "bb43241e568340499fa930b7fa9ceed3"
format_stamp: "Formatted at 2026-08-11 12:46:41 on dist-test-slave-f0tn"
server_key: "faf35bdc111751767b4fe7265608da05"
server_key_iv: "5968cc7ccde0a3d4813afdeb554d7ba0"
server_key_version: "encryptionkey@0"
I20260811 12:46:41.039691 12213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "bb43241e568340499fa930b7fa9ceed3"
format_stamp: "Formatted at 2026-08-11 12:46:41 on dist-test-slave-f0tn"
server_key: "faf35bdc111751767b4fe7265608da05"
server_key_iv: "5968cc7ccde0a3d4813afdeb554d7ba0"
server_key_version: "encryptionkey@0"
I20260811 12:46:41.044996 12213 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:46:41.048406 12243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:41.049727 12213 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:46:41.049878 12213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "bb43241e568340499fa930b7fa9ceed3"
format_stamp: "Formatted at 2026-08-11 12:46:41 on dist-test-slave-f0tn"
server_key: "faf35bdc111751767b4fe7265608da05"
server_key_iv: "5968cc7ccde0a3d4813afdeb554d7ba0"
server_key_version: "encryptionkey@0"
I20260811 12:46:41.050061 12213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:41.069324 12213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:41.072935 12213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:41.073196 12213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:41.074141 12213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:46:41.075547 12213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:41.075620 12213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:41.075735 12213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:41.075784 12213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:41.105549 12213 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:42055
I20260811 12:46:41.105568 12439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:42055 every 8 connection(s)
I20260811 12:46:41.106812 12213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:46:41.115427 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 12213
I20260811 12:46:41.115604 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:46:41.116005 7613 external_mini_cluster.cc:1428] Setting key d0d971f63b3d7b5c5165cd0c7c22f02f
Aug 11 12:46:41 dist-test-slave-f0tn krb5kdc[11778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452401, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:41.126433 12205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:41.109348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:49097 (local address 127.7.111.126:37519)
0811 12:46:41.109548 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:41.109553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:41.110805 (+ 1252us) server_negotiation.cc:407] Connection header received
0811 12:46:41.112644 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:41.112650 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:41.112735 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:41.112824 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:41.114396 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:41.115097 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:41.116803 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:41.117335 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:41.117467 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:41.121014 (+ 3547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:41.121019 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:41.121022 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:41.121054 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:41.123488 (+ 2434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:41.124181 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:41.124185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:41.124187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:41.124315 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:41.124733 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:41.124736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:41.124738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:41.125012 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:41.125145 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:41.125855 (+ 710us) server_negotiation.cc:299] Negotiation successful
0811 12:46:41.126086 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 12:46:41.127354 12443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:41.109811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:37519 (local address 127.7.111.66:49097)
0811 12:46:41.110625 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:41.110685 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:41.112373 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:41.113074 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:41.113083 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:41.113650 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:41.114221 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:41.114239 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:41.115260 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:41.115265 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:41.116654 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:41.116680 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:41.117611 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:41.117619 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:41.117925 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:41.118612 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:41.118639 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:41.120802 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:41.123618 (+ 2816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:41.123627 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:41.123644 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:41.124056 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:41.124422 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:41.124427 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:41.124431 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:41.124615 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:41.125137 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:41.125145 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:41.125493 (+ 348us) client_negotiation.cc:769] Sending connection context
0811 12:46:41.125749 (+ 256us) client_negotiation.cc:240] Negotiation successful
0811 12:46:41.126259 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":296,"threads_started":1}
I20260811 12:46:41.128721 12440 heartbeater.cc:347] Connected to a master server at 127.7.111.126:37519
I20260811 12:46:41.129040 12440 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:41.129982 12440 heartbeater.cc:511] Master 127.7.111.126:37519 requested a full tablet report, sending...
I20260811 12:46:41.131536 11853 ts_manager.cc:194] Registered new tserver with Master: bb43241e568340499fa930b7fa9ceed3 (127.7.111.66:42055)
I20260811 12:46:41.132344 11853 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:49097
WARNING: no policy specified for kudu/127.7.111.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
I20260811 12:46:41.166440 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:37519
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:36183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:41.492913 12446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:41.493177 12446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:41.493324 12446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:41.500324 12446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:41.500434 12446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:41.500591 12446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:46:41.510352 12446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:36183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:37519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:41.511592 12446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:41.512914 12446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:41.525535 12453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:41.525658 12455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:41.525746 12458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:41.526620 12446 server_base.cc:1034] running on GCE node
I20260811 12:46:41.527154 12446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:41.528015 12446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:41.529208 12446 hybrid_clock.cc:648] HybridClock initialized: now 1786452401529155 us; error 42 us; skew 500 ppm
I20260811 12:46:41.529515 12446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:41 dist-test-slave-f0tn krb5kdc[11778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452401, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:41.532572 12446 init.cc:377] Logged in from keytab as kudu/127.7.111.67@KRBTEST.COM (short username kudu)
I20260811 12:46:41.533979 12446 webserver.cc:460] Webserver started at http://127.7.111.67:34917/ using document root <none> and password file <none>
I20260811 12:46:41.534626 12446 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:41.534751 12446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:41.535072 12446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:41.538354 12446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "87789ab8d47c4b68a999ec73b2a35bd4"
format_stamp: "Formatted at 2026-08-11 12:46:41 on dist-test-slave-f0tn"
server_key: "2089e3b63a5d4cf4ef07016c89b7c6b7"
server_key_iv: "16e387ca10571bfe9498637f7c2ce6d3"
server_key_version: "encryptionkey@0"
I20260811 12:46:41.539354 12446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "87789ab8d47c4b68a999ec73b2a35bd4"
format_stamp: "Formatted at 2026-08-11 12:46:41 on dist-test-slave-f0tn"
server_key: "2089e3b63a5d4cf4ef07016c89b7c6b7"
server_key_iv: "16e387ca10571bfe9498637f7c2ce6d3"
server_key_version: "encryptionkey@0"
I20260811 12:46:41.546762 12446 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.004s
I20260811 12:46:41.550405 12472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:41.551685 12446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:46:41.551824 12446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "87789ab8d47c4b68a999ec73b2a35bd4"
format_stamp: "Formatted at 2026-08-11 12:46:41 on dist-test-slave-f0tn"
server_key: "2089e3b63a5d4cf4ef07016c89b7c6b7"
server_key_iv: "16e387ca10571bfe9498637f7c2ce6d3"
server_key_version: "encryptionkey@0"
I20260811 12:46:41.551999 12446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:41.567683 12446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:41.571312 12446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:41.571545 12446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:41.572430 12446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:46:41.573738 12446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:41.573812 12446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:41.573901 12446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:41.573952 12446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:41.604888 12446 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:34281
I20260811 12:46:41.604923 12685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:34281 every 8 connection(s)
I20260811 12:46:41.606149 12446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:46:41.616499 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 12446
I20260811 12:46:41.616693 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:46:41.617192 7613 external_mini_cluster.cc:1428] Setting key 0aa3c99c107766dec52d2b46a39dec9d
Aug 11 12:46:41 dist-test-slave-f0tn krb5kdc[11778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452401, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:41.625340 12205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:41.608506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:55805 (local address 127.7.111.126:37519)
0811 12:46:41.608732 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:41.608738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:41.610057 (+ 1319us) server_negotiation.cc:407] Connection header received
0811 12:46:41.611874 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:41.611880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:41.611981 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:41.612097 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:41.613511 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:41.614270 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:41.615792 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:41.616241 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:41.616385 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:41.620071 (+ 3686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:41.620078 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:41.620082 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:41.620123 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:41.622492 (+ 2369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:41.623164 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:41.623168 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:41.623170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:41.623307 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:41.623724 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:41.623728 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:41.623730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:41.623992 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:41.624125 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:41.624789 (+ 664us) server_negotiation.cc:299] Negotiation successful
0811 12:46:41.625006 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:46:41.626183 12688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:41.609064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:37519 (local address 127.7.111.67:55805)
0811 12:46:41.609885 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:41.609929 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:41.611607 (+ 1678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:41.612283 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:41.612290 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:41.612784 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:41.613328 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:41.613346 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:41.614411 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:41.614415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:41.615665 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:41.615678 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:41.616434 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:41.616438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:41.616646 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:41.617363 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:41.617420 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:41.619878 (+ 2458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:41.622622 (+ 2744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:41.622630 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:41.622645 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:41.623045 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:41.623413 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:41.623418 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:41.623422 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:41.623612 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:41.624116 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:41.624123 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:41.624425 (+ 302us) client_negotiation.cc:769] Sending connection context
0811 12:46:41.624644 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 12:46:41.625120 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":339,"threads_started":1}
I20260811 12:46:41.627364 12686 heartbeater.cc:347] Connected to a master server at 127.7.111.126:37519
I20260811 12:46:41.627643 12686 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:41.628350 12686 heartbeater.cc:511] Master 127.7.111.126:37519 requested a full tablet report, sending...
I20260811 12:46:41.629691 11853 ts_manager.cc:194] Registered new tserver with Master: 87789ab8d47c4b68a999ec73b2a35bd4 (127.7.111.67:34281)
I20260811 12:46:41.630452 11853 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:55805
I20260811 12:46:41.632237 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:46:41.643783 12202 heartbeater.cc:503] Master 127.7.111.126:37519 was elected leader, sending a full tablet report...
W20260811 12:46:41.670141 12205 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:46:41.667805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43780 (local address 127.7.111.126:37519)
0811 12:46:41.667984 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:41.667988 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:41.668471 (+ 483us) server_negotiation.cc:407] Connection header received
0811 12:46:41.668626 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:41.668630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:41.668698 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:41.668792 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:41.669949 (+ 1157us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43780: BlockingRecv error: recv got EOF from 127.0.0.1:43780 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:46:41.673058 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 11956
W20260811 12:46:41.675681 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 11956: Not authorized: Client connection negotiation failed: client connection to 127.7.111.65:39473: server requires authentication, but client does not have Kerberos credentials available
I20260811 12:46:41.675865 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 11956
I20260811 12:46:41.700366 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 12213
W20260811 12:46:41.702989 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 12213: Not authorized: Client connection negotiation failed: client connection to 127.7.111.66:42055: server requires authentication, but client does not have Kerberos credentials available
I20260811 12:46:41.703171 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 12213
I20260811 12:46:41.726048 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 12446
W20260811 12:46:41.728878 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 12446: Not authorized: Client connection negotiation failed: client connection to 127.7.111.67:34281: server requires authentication, but client does not have Kerberos credentials available
I20260811 12:46:41.728988 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 12446
I20260811 12:46:41.756632 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 11796
I20260811 12:46:41.757835 11909 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:41.862771 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 11796
2026-08-11T12:46:41Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3691 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:46:41 dist-test-slave-f0tn krb5kdc[12724](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:46:41 dist-test-slave-f0tn krb5kdc[12724](info): set up 2 sockets
Aug 11 12:46:41 dist-test-slave-f0tn krb5kdc[12724](info): commencing operation
krb5kdc: starting...
W20260811 12:46:43.128031 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:46:43 dist-test-slave-f0tn krb5kdc[12724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:46:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:46:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:46:43.341740 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:36739
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:45617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:36739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:43.664239 12747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:43.664515 12747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:43.664613 12747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:43.671707 12747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:46:43.671811 12747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:43.671865 12747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:43.671921 12747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:46:43.671967 12747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:46:43.681569 12747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:45617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:36739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:36739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:43.682791 12747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:43.684038 12747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:43.695372 12753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:43.695408 12754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:43.695438 12758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:43.696324 12747 server_base.cc:1034] running on GCE node
I20260811 12:46:43.697070 12747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:43.698412 12747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:43.699597 12747 hybrid_clock.cc:648] HybridClock initialized: now 1786452403699554 us; error 36 us; skew 500 ppm
I20260811 12:46:43.699923 12747 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:43 dist-test-slave-f0tn krb5kdc[12724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452403, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:43.702889 12747 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:46:43.704291 12747 webserver.cc:460] Webserver started at http://127.7.111.126:45155/ using document root <none> and password file <none>
I20260811 12:46:43.704969 12747 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:43.705081 12747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:43.705411 12747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:43.708285 12747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "92ce49ed100846c2b62458005bbb52a4"
format_stamp: "Formatted at 2026-08-11 12:46:43 on dist-test-slave-f0tn"
server_key: "363007d0d74d4ec8cfadb894284d7504"
server_key_iv: "5b8d51f3742077258e128959c7e07bf2"
server_key_version: "encryptionkey@0"
I20260811 12:46:43.709041 12747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "92ce49ed100846c2b62458005bbb52a4"
format_stamp: "Formatted at 2026-08-11 12:46:43 on dist-test-slave-f0tn"
server_key: "363007d0d74d4ec8cfadb894284d7504"
server_key_iv: "5b8d51f3742077258e128959c7e07bf2"
server_key_version: "encryptionkey@0"
I20260811 12:46:43.713971 12747 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260811 12:46:43.717231 12769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:43.718525 12747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:46:43.718696 12747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "92ce49ed100846c2b62458005bbb52a4"
format_stamp: "Formatted at 2026-08-11 12:46:43 on dist-test-slave-f0tn"
server_key: "363007d0d74d4ec8cfadb894284d7504"
server_key_iv: "5b8d51f3742077258e128959c7e07bf2"
server_key_version: "encryptionkey@0"
I20260811 12:46:43.718874 12747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:43.730278 12747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:43.733914 12747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:43.734169 12747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:43.751199 12747 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:36739
I20260811 12:46:43.751232 12858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:36739 every 8 connection(s)
I20260811 12:46:43.752566 12747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:46:43.756191 12859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:43.759246 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 12747
I20260811 12:46:43.759455 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:46:43.760248 7613 external_mini_cluster.cc:1428] Setting key 1c1a2dfafd6764e2e58792be02675f2e
I20260811 12:46:43.768417 12859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92ce49ed100846c2b62458005bbb52a4: Bootstrap starting.
I20260811 12:46:43.772030 12859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92ce49ed100846c2b62458005bbb52a4: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:43.773306 12859 log.cc:824] T 00000000000000000000000000000000 P 92ce49ed100846c2b62458005bbb52a4: Log is configured to *not* fsync() on all Append() calls
Aug 11 12:46:43 dist-test-slave-f0tn krb5kdc[12724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:43.776384 12859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92ce49ed100846c2b62458005bbb52a4: No bootstrap required, opened a new log
I20260811 12:46:43.783780 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:43.762133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48038 (local address 127.7.111.126:36739)
0811 12:46:43.762900 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:43.762912 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:43.762952 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:46:43.764534 (+ 1582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:43.764558 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:43.764958 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:43.765277 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:43.767113 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:43.768381 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:43.770074 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:43.770725 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:43.770846 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:43.775505 (+ 4659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:43.775515 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:43.775535 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:43.775580 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:43.779403 (+ 3823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:43.780030 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:43.780039 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:43.780047 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:43.780264 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:43.780595 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:43.780601 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:43.780604 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:43.781288 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:43.781638 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:43.781997 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 12:46:43.782418 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":351,"threads_started":1}
I20260811 12:46:43.786644 12859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92ce49ed100846c2b62458005bbb52a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92ce49ed100846c2b62458005bbb52a4" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36739 } }
I20260811 12:46:43.787056 12859 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92ce49ed100846c2b62458005bbb52a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:43.787163 12859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92ce49ed100846c2b62458005bbb52a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92ce49ed100846c2b62458005bbb52a4, State: Initialized, Role: FOLLOWER
I20260811 12:46:43.787607 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:46:43.787783 12859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92ce49ed100846c2b62458005bbb52a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92ce49ed100846c2b62458005bbb52a4" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36739 } }
I20260811 12:46:43.788036 12859 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92ce49ed100846c2b62458005bbb52a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:46:43.788158 12859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92ce49ed100846c2b62458005bbb52a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:46:43.788326 12859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92ce49ed100846c2b62458005bbb52a4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:43.791426 12859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92ce49ed100846c2b62458005bbb52a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92ce49ed100846c2b62458005bbb52a4" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36739 } }
I20260811 12:46:43.791994 12859 leader_election.cc:302] T 00000000000000000000000000000000 P 92ce49ed100846c2b62458005bbb52a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92ce49ed100846c2b62458005bbb52a4; no voters:
I20260811 12:46:43.792611 12859 leader_election.cc:288] T 00000000000000000000000000000000 P 92ce49ed100846c2b62458005bbb52a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:46:43.792701 12865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92ce49ed100846c2b62458005bbb52a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:46:43.793874 12865 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92ce49ed100846c2b62458005bbb52a4 [term 1 LEADER]: Becoming Leader. State: Replica: 92ce49ed100846c2b62458005bbb52a4, State: Running, Role: LEADER
I20260811 12:46:43.794399 12865 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92ce49ed100846c2b62458005bbb52a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92ce49ed100846c2b62458005bbb52a4" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36739 } }
I20260811 12:46:43.794782 12859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92ce49ed100846c2b62458005bbb52a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:46:43.796955 12867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92ce49ed100846c2b62458005bbb52a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92ce49ed100846c2b62458005bbb52a4. Latest consensus state: current_term: 1 leader_uuid: "92ce49ed100846c2b62458005bbb52a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92ce49ed100846c2b62458005bbb52a4" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36739 } } }
I20260811 12:46:43.797305 12867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92ce49ed100846c2b62458005bbb52a4 [sys.catalog]: This master's current role is: LEADER
I20260811 12:46:43.797878 12866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92ce49ed100846c2b62458005bbb52a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92ce49ed100846c2b62458005bbb52a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92ce49ed100846c2b62458005bbb52a4" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36739 } } }
I20260811 12:46:43.798174 12866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92ce49ed100846c2b62458005bbb52a4 [sys.catalog]: This master's current role is: LEADER
I20260811 12:46:43.800225 12875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:46:43.805035 12875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:46:43.821988 12875 catalog_manager.cc:1347] Generated new cluster ID: 94dd8338aa4d43b9a03c124b8227f502
I20260811 12:46:43.822118 12875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:46:43.840521 12875 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:46:43.842244 12875 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:46:43.851861 12875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92ce49ed100846c2b62458005bbb52a4: Generated new TSK 0
I20260811 12:46:43.852952 12875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:46:43.853783 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:43.843427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48048 (local address 127.7.111.126:36739)
0811 12:46:43.843603 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:43.843609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:43.844253 (+ 644us) server_negotiation.cc:407] Connection header received
0811 12:46:43.844445 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:43.844451 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:43.844531 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:43.844630 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:43.845885 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:43.846953 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:43.848819 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:43.849463 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:43.849598 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:43.851808 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:43.851815 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:43.851820 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:46:43.852798 (+ 978us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:43.852889 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:46:43.852932 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:43.853024 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:43.853273 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 12:46:43.853410 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:46:43.860746 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 12747
I20260811 12:46:43.861868 12849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:43.949749 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 12747
2026-08-11T12:46:43Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2080 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:46:44 dist-test-slave-f0tn krb5kdc[12911](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:46:44 dist-test-slave-f0tn krb5kdc[12911](info): set up 2 sockets
Aug 11 12:46:44 dist-test-slave-f0tn krb5kdc[12911](info): commencing operation
krb5kdc: starting...
W20260811 12:46:45.168206 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:46:45 dist-test-slave-f0tn krb5kdc[12911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:46:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:46:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:46:45.378404 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:34209
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:40963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:34209
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:45.704213 12936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:45.704491 12936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:45.704588 12936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:45.711607 12936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:46:45.711697 12936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:45.711755 12936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:45.711812 12936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:46:45.711853 12936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:46:45.721165 12936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:40963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:34209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:34209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:45.722385 12936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:45.723598 12936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:45.734864 12941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:45.734804 12946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:45.734853 12943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:45.735757 12936 server_base.cc:1034] running on GCE node
I20260811 12:46:45.736522 12936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:45.737999 12936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:45.739210 12936 hybrid_clock.cc:648] HybridClock initialized: now 1786452405739143 us; error 48 us; skew 500 ppm
I20260811 12:46:45.739537 12936 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:45 dist-test-slave-f0tn krb5kdc[12911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452405, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:45.742604 12936 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:46:45.744083 12936 webserver.cc:460] Webserver started at http://127.7.111.126:36733/ using document root <none> and password file <none>
I20260811 12:46:45.744735 12936 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:45.744856 12936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:45.745151 12936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:45.748148 12936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "41fa2680ef0c40ce8aa0360098181f8c"
format_stamp: "Formatted at 2026-08-11 12:46:45 on dist-test-slave-f0tn"
server_key: "fffc2b79d6b92d88e36a84125a5d6d00"
server_key_iv: "4816afe1e8c0afd80038b5e2dabfa1b8"
server_key_version: "encryptionkey@0"
I20260811 12:46:45.748912 12936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "41fa2680ef0c40ce8aa0360098181f8c"
format_stamp: "Formatted at 2026-08-11 12:46:45 on dist-test-slave-f0tn"
server_key: "fffc2b79d6b92d88e36a84125a5d6d00"
server_key_iv: "4816afe1e8c0afd80038b5e2dabfa1b8"
server_key_version: "encryptionkey@0"
I20260811 12:46:45.753895 12936 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:46:45.757184 12954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:45.758520 12936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:46:45.758683 12936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "41fa2680ef0c40ce8aa0360098181f8c"
format_stamp: "Formatted at 2026-08-11 12:46:45 on dist-test-slave-f0tn"
server_key: "fffc2b79d6b92d88e36a84125a5d6d00"
server_key_iv: "4816afe1e8c0afd80038b5e2dabfa1b8"
server_key_version: "encryptionkey@0"
I20260811 12:46:45.758855 12936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:45.773900 12936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:45.778478 12936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:45.778716 12936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:45.796743 12936 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:34209
I20260811 12:46:45.796766 13045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:34209 every 8 connection(s)
I20260811 12:46:45.798183 12936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:46:45.799310 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 12936
I20260811 12:46:45.799495 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:46:45.800149 7613 external_mini_cluster.cc:1428] Setting key d5d60153fc9307a2c940ae387077472a
I20260811 12:46:45.803221 13047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 12:46:45 dist-test-slave-f0tn krb5kdc[12911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:45.815372 13047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41fa2680ef0c40ce8aa0360098181f8c: Bootstrap starting.
I20260811 12:46:45.819281 13047 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41fa2680ef0c40ce8aa0360098181f8c: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:45.820425 13047 log.cc:824] T 00000000000000000000000000000000 P 41fa2680ef0c40ce8aa0360098181f8c: Log is configured to *not* fsync() on all Append() calls
I20260811 12:46:45.823875 13047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41fa2680ef0c40ce8aa0360098181f8c: No bootstrap required, opened a new log
I20260811 12:46:45.824153 13050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:45.802240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54506 (local address 127.7.111.126:34209)
0811 12:46:45.803061 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:45.803076 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:45.803125 (+ 49us) server_negotiation.cc:407] Connection header received
0811 12:46:45.805312 (+ 2187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:45.805341 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:45.805850 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:45.806292 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:45.807527 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:45.809544 (+ 2017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:45.810713 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:45.811684 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:45.811869 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:45.815950 (+ 4081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:45.815961 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:45.815978 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:45.816022 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:45.819780 (+ 3758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:45.820474 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:45.820482 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:45.820491 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:45.820692 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:45.821111 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:45.821115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:45.821118 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:45.821688 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:45.822071 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:45.822419 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 12:46:45.822834 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":322,"threads_started":1}
I20260811 12:46:45.831383 13047 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41fa2680ef0c40ce8aa0360098181f8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41fa2680ef0c40ce8aa0360098181f8c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34209 } }
I20260811 12:46:45.831811 13047 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41fa2680ef0c40ce8aa0360098181f8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:45.831912 13047 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41fa2680ef0c40ce8aa0360098181f8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41fa2680ef0c40ce8aa0360098181f8c, State: Initialized, Role: FOLLOWER
I20260811 12:46:45.832500 13047 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41fa2680ef0c40ce8aa0360098181f8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41fa2680ef0c40ce8aa0360098181f8c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34209 } }
I20260811 12:46:45.832722 13047 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41fa2680ef0c40ce8aa0360098181f8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:46:45.832847 13047 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41fa2680ef0c40ce8aa0360098181f8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:46:45.833006 13047 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41fa2680ef0c40ce8aa0360098181f8c [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:45.835106 13047 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41fa2680ef0c40ce8aa0360098181f8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41fa2680ef0c40ce8aa0360098181f8c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34209 } }
I20260811 12:46:45.835564 13047 leader_election.cc:302] T 00000000000000000000000000000000 P 41fa2680ef0c40ce8aa0360098181f8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41fa2680ef0c40ce8aa0360098181f8c; no voters:
I20260811 12:46:45.836126 13047 leader_election.cc:288] T 00000000000000000000000000000000 P 41fa2680ef0c40ce8aa0360098181f8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:46:45.836239 13052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41fa2680ef0c40ce8aa0360098181f8c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:46:45.836851 13052 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41fa2680ef0c40ce8aa0360098181f8c [term 1 LEADER]: Becoming Leader. State: Replica: 41fa2680ef0c40ce8aa0360098181f8c, State: Running, Role: LEADER
I20260811 12:46:45.837334 13052 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41fa2680ef0c40ce8aa0360098181f8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41fa2680ef0c40ce8aa0360098181f8c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34209 } }
I20260811 12:46:45.838073 13047 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41fa2680ef0c40ce8aa0360098181f8c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:46:45.840044 13055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41fa2680ef0c40ce8aa0360098181f8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41fa2680ef0c40ce8aa0360098181f8c. Latest consensus state: current_term: 1 leader_uuid: "41fa2680ef0c40ce8aa0360098181f8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41fa2680ef0c40ce8aa0360098181f8c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34209 } } }
I20260811 12:46:45.840116 13054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41fa2680ef0c40ce8aa0360098181f8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41fa2680ef0c40ce8aa0360098181f8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41fa2680ef0c40ce8aa0360098181f8c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34209 } } }
I20260811 12:46:45.840459 13055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41fa2680ef0c40ce8aa0360098181f8c [sys.catalog]: This master's current role is: LEADER
I20260811 12:46:45.840567 13054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41fa2680ef0c40ce8aa0360098181f8c [sys.catalog]: This master's current role is: LEADER
I20260811 12:46:45.841462 13064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:46:45.847426 13064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:46:45.856923 13064 catalog_manager.cc:1347] Generated new cluster ID: c2f69b00432243929d2ea0d3e5cca046
I20260811 12:46:45.857053 13064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:46:45.909839 13064 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:46:45.911283 13064 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:46:45.917574 13064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41fa2680ef0c40ce8aa0360098181f8c: Generated new TSK 0
I20260811 12:46:45.918391 13064 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.111.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
I20260811 12:46:45.982692 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34209
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:40963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:46.378727 13082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:46.378984 13082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:46.379106 13082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:46.386119 13082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:46.386214 13082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:46.386358 13082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:46:46.395766 13082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:40963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:46.396986 13082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:46.398324 13082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:46.410907 13088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:46.410934 13089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:46.410866 13091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:46.411653 13082 server_base.cc:1034] running on GCE node
I20260811 12:46:46.412204 13082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:46.413085 13082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:46.414286 13082 hybrid_clock.cc:648] HybridClock initialized: now 1786452406414237 us; error 38 us; skew 500 ppm
I20260811 12:46:46.414595 13082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:46 dist-test-slave-f0tn krb5kdc[12911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452406, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:46.420194 13082 init.cc:377] Logged in from keytab as kudu/127.7.111.65@KRBTEST.COM (short username kudu)
I20260811 12:46:46.421671 13082 webserver.cc:460] Webserver started at http://127.7.111.65:33283/ using document root <none> and password file <none>
I20260811 12:46:46.422348 13082 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:46.422465 13082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:46.422766 13082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:46.425757 13082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "1b4b6a3117134fb886b4cdeab5ab4e53"
format_stamp: "Formatted at 2026-08-11 12:46:46 on dist-test-slave-f0tn"
server_key: "e4314a62da7a455f31d5a9a2186421c1"
server_key_iv: "00efef2c757e2ca527639401b379e85c"
server_key_version: "encryptionkey@0"
I20260811 12:46:46.426509 13082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "1b4b6a3117134fb886b4cdeab5ab4e53"
format_stamp: "Formatted at 2026-08-11 12:46:46 on dist-test-slave-f0tn"
server_key: "e4314a62da7a455f31d5a9a2186421c1"
server_key_iv: "00efef2c757e2ca527639401b379e85c"
server_key_version: "encryptionkey@0"
I20260811 12:46:46.431356 13082 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:46:46.434633 13107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:46.435840 13082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:46:46.435982 13082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "1b4b6a3117134fb886b4cdeab5ab4e53"
format_stamp: "Formatted at 2026-08-11 12:46:46 on dist-test-slave-f0tn"
server_key: "e4314a62da7a455f31d5a9a2186421c1"
server_key_iv: "00efef2c757e2ca527639401b379e85c"
server_key_version: "encryptionkey@0"
I20260811 12:46:46.436148 13082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:46.446254 13082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:46.449786 13082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:46.450053 13082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:46.450901 13082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:46:46.452114 13082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:46.452188 13082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:46.452308 13082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:46.452361 13082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:46.482883 13082 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:36509
I20260811 12:46:46.482933 13308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:36509 every 8 connection(s)
I20260811 12:46:46.484103 13082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:46:46.488217 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 13082
I20260811 12:46:46.488426 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:46:46.489305 7613 external_mini_cluster.cc:1428] Setting key ce1b6048f0506f751bff8388324e0beb
Aug 11 12:46:46 dist-test-slave-f0tn krb5kdc[12911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452406, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:46.506206 13312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:46.486631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:40839 (local address 127.7.111.126:34209)
0811 12:46:46.487108 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:46.487114 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:46.488136 (+ 1022us) server_negotiation.cc:407] Connection header received
0811 12:46:46.490514 (+ 2378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:46.490518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:46.490591 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:46.490673 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:46.492576 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:46.493324 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:46.495430 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:46.495830 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:46.495927 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:46.499899 (+ 3972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:46.499904 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:46.499907 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:46.499939 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:46.502990 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:46.503734 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:46.503740 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:46.503743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:46.503916 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:46.504342 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:46.504346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:46.504349 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:46.504683 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:46.504822 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:46.505626 (+ 804us) server_negotiation.cc:299] Negotiation successful
0811 12:46:46.505844 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":219,"threads_started":1}
I20260811 12:46:46.506978 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:46.487192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34209 (local address 127.7.111.65:40839)
0811 12:46:46.487947 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:46.488007 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:46.490236 (+ 2229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:46.490944 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:46.490955 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:46.491585 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:46.492385 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:46.492410 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:46.493496 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:46.493501 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:46.495290 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:46.495306 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:46.496057 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:46.496064 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:46.496344 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:46.497028 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:46.497058 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:46.499673 (+ 2615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:46.503149 (+ 3476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:46.503157 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:46.503171 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:46.503609 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:46.504045 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:46.504049 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:46.504051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:46.504239 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:46.504826 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:46.504832 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:46.505188 (+ 356us) client_negotiation.cc:769] Sending connection context
0811 12:46:46.505469 (+ 281us) client_negotiation.cc:240] Negotiation successful
0811 12:46:46.505877 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":259,"threads_started":1}
I20260811 12:46:46.508540 13309 heartbeater.cc:347] Connected to a master server at 127.7.111.126:34209
I20260811 12:46:46.508859 13309 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:46.509627 13309 heartbeater.cc:511] Master 127.7.111.126:34209 requested a full tablet report, sending...
I20260811 12:46:46.512081 12989 ts_manager.cc:194] Registered new tserver with Master: 1b4b6a3117134fb886b4cdeab5ab4e53 (127.7.111.65:36509)
WARNING: no policy specified for kudu/127.7.111.66@KRBTEST.COM; defaulting to no policy
I20260811 12:46:46.514382 12989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:40839
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
I20260811 12:46:46.540503 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34209
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:40963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:46.865348 13321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:46.865633 13321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:46.865778 13321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:46.872740 13321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:46.872833 13321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:46.872969 13321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:46:46.882294 13321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:40963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:46.883544 13321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:46.884840 13321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:46.897332 13333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:46.897344 13329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:46.897403 13330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:46.898268 13321 server_base.cc:1034] running on GCE node
I20260811 12:46:46.898860 13321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:46.899734 13321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:46.900935 13321 hybrid_clock.cc:648] HybridClock initialized: now 1786452406900890 us; error 37 us; skew 500 ppm
I20260811 12:46:46.901239 13321 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:46 dist-test-slave-f0tn krb5kdc[12911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452406, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:46.904247 13321 init.cc:377] Logged in from keytab as kudu/127.7.111.66@KRBTEST.COM (short username kudu)
I20260811 12:46:46.905728 13321 webserver.cc:460] Webserver started at http://127.7.111.66:44147/ using document root <none> and password file <none>
I20260811 12:46:46.906390 13321 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:46.906518 13321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:46.906831 13321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:46.909754 13321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "251a5efda5b847d6a108d523c96ff0cf"
format_stamp: "Formatted at 2026-08-11 12:46:46 on dist-test-slave-f0tn"
server_key: "5beaf1a193be9e183c5f28a7634e24bd"
server_key_iv: "087ff4d1b83fc50c5ab9710b53fdab98"
server_key_version: "encryptionkey@0"
I20260811 12:46:46.910521 13321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "251a5efda5b847d6a108d523c96ff0cf"
format_stamp: "Formatted at 2026-08-11 12:46:46 on dist-test-slave-f0tn"
server_key: "5beaf1a193be9e183c5f28a7634e24bd"
server_key_iv: "087ff4d1b83fc50c5ab9710b53fdab98"
server_key_version: "encryptionkey@0"
I20260811 12:46:46.915498 13321 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 12:46:46.918906 13355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:46.920116 13321 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:46:46.920276 13321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "251a5efda5b847d6a108d523c96ff0cf"
format_stamp: "Formatted at 2026-08-11 12:46:46 on dist-test-slave-f0tn"
server_key: "5beaf1a193be9e183c5f28a7634e24bd"
server_key_iv: "087ff4d1b83fc50c5ab9710b53fdab98"
server_key_version: "encryptionkey@0"
I20260811 12:46:46.920457 13321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:46.951725 13321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:46.955540 13321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:46.955811 13321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:46.956763 13321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:46:46.958093 13321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:46.958180 13321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:46.958307 13321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:46.958361 13321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:46.989269 13321 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:46617
I20260811 12:46:46.989337 13561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:46617 every 8 connection(s)
I20260811 12:46:46.990590 13321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:46:46.999344 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 13321
I20260811 12:46:46.999526 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:46:47.000131 7613 external_mini_cluster.cc:1428] Setting key 71c0db8bb994b4321675028d49640e97
Aug 11 12:46:47 dist-test-slave-f0tn krb5kdc[12911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452406, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:47.011250 13312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:46.993066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:51633 (local address 127.7.111.126:34209)
0811 12:46:46.993244 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:46.993248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:46.994457 (+ 1209us) server_negotiation.cc:407] Connection header received
0811 12:46:46.996330 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:46.996335 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:46.996410 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:46.996497 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:46.997953 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:46.998684 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:47.000829 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:47.001439 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:47.001563 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:47.005509 (+ 3946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:47.005514 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:47.005517 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:47.005551 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:47.008184 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:47.008979 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:47.008983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:47.008985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:47.009119 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:47.009608 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:47.009612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:47.009614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:47.009874 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:47.009982 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:47.010752 (+ 770us) server_negotiation.cc:299] Negotiation successful
0811 12:46:47.010919 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 12:46:47.012279 13567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:46.993582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34209 (local address 127.7.111.66:51633)
0811 12:46:46.994293 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:46.994330 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:46.996068 (+ 1738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:46.996683 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:46.996690 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:46.997176 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:46.997794 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:46.997812 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:46.998805 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:46.998808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:47.000680 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:47.000695 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:47.001644 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:47.001649 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:47.001871 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:47.002563 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:47.002601 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:47.005238 (+ 2637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:47.008322 (+ 3084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:47.008332 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:47.008349 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:47.008864 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:47.009246 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:47.009251 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:47.009254 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:47.009519 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:47.009989 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:47.009996 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:47.010403 (+ 407us) client_negotiation.cc:769] Sending connection context
0811 12:46:47.010684 (+ 281us) client_negotiation.cc:240] Negotiation successful
0811 12:46:47.011168 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":277,"threads_started":1}
I20260811 12:46:47.013655 13563 heartbeater.cc:347] Connected to a master server at 127.7.111.126:34209
I20260811 12:46:47.013999 13563 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:47.014935 13563 heartbeater.cc:511] Master 127.7.111.126:34209 requested a full tablet report, sending...
I20260811 12:46:47.016449 12989 ts_manager.cc:194] Registered new tserver with Master: 251a5efda5b847d6a108d523c96ff0cf (127.7.111.66:46617)
I20260811 12:46:47.017221 12989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:51633
WARNING: no policy specified for kudu/127.7.111.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
I20260811 12:46:47.049929 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34209
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:40963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:47.376138 13573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:47.376423 13573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:47.376528 13573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:47.383725 13573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:47.383842 13573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:47.383957 13573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:46:47.393677 13573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:40963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:47.394984 13573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:47.396317 13573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:47.409245 13581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:47.409335 13588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:47.409438 13582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:47.410463 13573 server_base.cc:1034] running on GCE node
I20260811 12:46:47.411027 13573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:47.411947 13573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:47.413143 13573 hybrid_clock.cc:648] HybridClock initialized: now 1786452407413086 us; error 48 us; skew 500 ppm
I20260811 12:46:47.413504 13573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:47 dist-test-slave-f0tn krb5kdc[12911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452407, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:47.416585 13573 init.cc:377] Logged in from keytab as kudu/127.7.111.67@KRBTEST.COM (short username kudu)
I20260811 12:46:47.418192 13573 webserver.cc:460] Webserver started at http://127.7.111.67:37653/ using document root <none> and password file <none>
I20260811 12:46:47.418924 13573 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:47.419045 13573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:47.419351 13573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:47.422343 13573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "7afea91113054c49bb784ba4968fb8b1"
format_stamp: "Formatted at 2026-08-11 12:46:47 on dist-test-slave-f0tn"
server_key: "55560d4800a0ead0416467f2d89dc920"
server_key_iv: "848bc8b6f491831a370a0c28b3711fef"
server_key_version: "encryptionkey@0"
I20260811 12:46:47.423120 13573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "7afea91113054c49bb784ba4968fb8b1"
format_stamp: "Formatted at 2026-08-11 12:46:47 on dist-test-slave-f0tn"
server_key: "55560d4800a0ead0416467f2d89dc920"
server_key_iv: "848bc8b6f491831a370a0c28b3711fef"
server_key_version: "encryptionkey@0"
I20260811 12:46:47.428257 13573 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:46:47.431641 13601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:47.432864 13573 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260811 12:46:47.433018 13573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "7afea91113054c49bb784ba4968fb8b1"
format_stamp: "Formatted at 2026-08-11 12:46:47 on dist-test-slave-f0tn"
server_key: "55560d4800a0ead0416467f2d89dc920"
server_key_iv: "848bc8b6f491831a370a0c28b3711fef"
server_key_version: "encryptionkey@0"
I20260811 12:46:47.433189 13573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:47.462265 13573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:47.466132 13573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:47.466408 13573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:47.467357 13573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:46:47.468709 13573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:47.468788 13573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:47.468909 13573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:47.468962 13573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:47.500283 13573 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:34509
I20260811 12:46:47.500330 13808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:34509 every 8 connection(s)
I20260811 12:46:47.501555 13573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:46:47.509310 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 13573
I20260811 12:46:47.509521 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:46:47.510141 7613 external_mini_cluster.cc:1428] Setting key 7f7c27622a8ac0fa6b4e4dd8f2b7e30a
Aug 11 12:46:47 dist-test-slave-f0tn krb5kdc[12911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452407, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:47.518118 13309 heartbeater.cc:503] Master 127.7.111.126:34209 was elected leader, sending a full tablet report...
I20260811 12:46:47.521307 13312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:47.503935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:39265 (local address 127.7.111.126:34209)
0811 12:46:47.504172 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:47.504179 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:47.505309 (+ 1130us) server_negotiation.cc:407] Connection header received
0811 12:46:47.507102 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:47.507107 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:47.507175 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:47.507262 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:47.508811 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:47.509606 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:47.511869 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:47.512275 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:47.512370 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:47.515970 (+ 3600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:47.515974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:47.515977 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:47.516011 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:47.518313 (+ 2302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:47.519083 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:47.519089 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:47.519092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:47.519239 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:47.519703 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:47.519707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:47.519709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:47.519965 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:47.520098 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:47.520757 (+ 659us) server_negotiation.cc:299] Negotiation successful
0811 12:46:47.520970 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 12:46:47.522260 13812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:47.504470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34209 (local address 127.7.111.67:39265)
0811 12:46:47.505152 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:47.505195 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:47.506837 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:47.507541 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:47.507550 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:47.508070 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:47.508668 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:47.508688 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:47.509773 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:47.509780 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:47.511726 (+ 1946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:47.511744 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:47.512507 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:47.512516 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:47.512727 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:47.513578 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:47.513605 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:47.515778 (+ 2173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:47.518448 (+ 2670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:47.518456 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:47.518471 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:47.518973 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:47.519346 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:47.519351 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:47.519355 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:47.519587 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:47.520092 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:47.520100 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:47.520408 (+ 308us) client_negotiation.cc:769] Sending connection context
0811 12:46:47.520633 (+ 225us) client_negotiation.cc:240] Negotiation successful
0811 12:46:47.521110 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":257,"threads_started":1}
I20260811 12:46:47.523517 13809 heartbeater.cc:347] Connected to a master server at 127.7.111.126:34209
I20260811 12:46:47.523859 13809 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:47.524735 13809 heartbeater.cc:511] Master 127.7.111.126:34209 requested a full tablet report, sending...
I20260811 12:46:47.526181 12989 ts_manager.cc:194] Registered new tserver with Master: 7afea91113054c49bb784ba4968fb8b1 (127.7.111.67:34509)
I20260811 12:46:47.526949 12989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:39265
I20260811 12:46:47.537532 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 12:46:47 dist-test-slave-f0tn krb5kdc[12911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452407, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 12:46:47 dist-test-slave-f0tn krb5kdc[12911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452407, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:47.593362 13312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:47.579956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54514 (local address 127.7.111.126:34209)
0811 12:46:47.580161 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:47.580167 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:47.580547 (+ 380us) server_negotiation.cc:407] Connection header received
0811 12:46:47.580745 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:47.580750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:47.580816 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:47.580902 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:47.582333 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:47.583090 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:47.584332 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:47.584758 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:47.584876 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:47.588491 (+ 3615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:47.588495 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:47.588498 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:47.588532 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:47.590848 (+ 2316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:47.591464 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:47.591468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:47.591470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:47.591602 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:47.591977 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:47.591981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:47.591983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:47.592316 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:47.592553 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:47.592836 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 12:46:47.593047 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 12:46:47.593972 12989 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54514
I20260811 12:46:47.595599 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 13082
Aug 11 12:46:47 dist-test-slave-f0tn krb5kdc[12911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452407, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.111.65@KRBTEST.COM
I20260811 12:46:47.610946 13829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:47.596553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56890 (local address 127.7.111.65:36509)
0811 12:46:47.597195 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:47.597202 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:47.597226 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:46:47.597282 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:47.597288 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:47.597586 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:47.597737 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:47.598911 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:47.599932 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:47.601112 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:47.601721 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:47.601848 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:47.605875 (+ 4027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:47.605887 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:47.605900 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:47.605936 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:47.608674 (+ 2738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:47.609290 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:47.609295 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:47.609298 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:47.609448 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:47.609744 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:47.609747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:47.609749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:47.610085 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:47.610298 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:47.610471 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 12:46:47.610659 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":340,"threads_started":1}
W20260811 12:46:47.612190 13304 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:56890
W20260811 12:46:47.613227 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 13082: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:46:47.613448 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 13082
I20260811 12:46:47.640571 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 13321
Aug 11 12:46:47 dist-test-slave-f0tn krb5kdc[12911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452407, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.111.66@KRBTEST.COM
I20260811 12:46:47.655494 13830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:47.641465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36066 (local address 127.7.111.66:46617)
0811 12:46:47.642079 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:47.642085 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:47.642112 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:46:47.642183 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:47.642189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:47.642401 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:47.642529 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:47.643766 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:47.644770 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:47.645948 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:47.646494 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:47.646589 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:47.650442 (+ 3853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:47.650454 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:47.650471 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:47.650505 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:47.653227 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:47.653811 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:47.653818 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:47.653820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:47.653943 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:47.654229 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:47.654231 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:47.654233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:47.654551 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:47.654774 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:47.654987 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 12:46:47.655182 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":325,"threads_started":1}
W20260811 12:46:47.656801 13549 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36066
W20260811 12:46:47.657913 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 13321: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:46:47.658053 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 13321
I20260811 12:46:47.684881 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 13573
Aug 11 12:46:47 dist-test-slave-f0tn krb5kdc[12911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452407, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.111.67@KRBTEST.COM
I20260811 12:46:47.700224 13833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:47.685765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51934 (local address 127.7.111.67:34509)
0811 12:46:47.686413 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:47.686420 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:47.686451 (+ 31us) server_negotiation.cc:407] Connection header received
0811 12:46:47.686514 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:47.686521 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:47.686745 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:47.686876 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:47.688097 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:47.689111 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:47.690411 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:47.690976 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:47.691074 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:47.695040 (+ 3966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:47.695053 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:47.695070 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:47.695111 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:47.697706 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:47.698350 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:47.698359 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:47.698361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:47.698525 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:47.698827 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:47.698831 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:47.698833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:47.699228 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:47.699489 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:47.699676 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 12:46:47.699902 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":291,"threads_started":1}
W20260811 12:46:47.701611 13799 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:51934
W20260811 12:46:47.702837 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 13573: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:46:47.703001 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 13573
I20260811 12:46:47.726569 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 12936
I20260811 12:46:47.727753 13035 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:47.830668 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 12936
2026-08-11T12:46:47Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3890 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:46:47 dist-test-slave-f0tn krb5kdc[13840](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:46:47 dist-test-slave-f0tn krb5kdc[13840](info): set up 2 sockets
Aug 11 12:46:47 dist-test-slave-f0tn krb5kdc[13840](info): commencing operation
krb5kdc: starting...
W20260811 12:46:49.060186 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:46:49 dist-test-slave-f0tn krb5kdc[13840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:46:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:46:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:46:49.277143 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:35367
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:43275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:35367
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:49.603065 13858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:49.603367 13858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:49.603513 13858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:49.610759 13858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:46:49.610857 13858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:49.610932 13858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:49.611008 13858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:46:49.611071 13858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:46:49.620782 13858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:43275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:35367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:35367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:49.622134 13858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:49.623476 13858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:49.634847 13863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:49.634806 13867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:49.634886 13865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:49.635967 13858 server_base.cc:1034] running on GCE node
I20260811 12:46:49.636730 13858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:49.638163 13858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:49.639389 13858 hybrid_clock.cc:648] HybridClock initialized: now 1786452409639336 us; error 40 us; skew 500 ppm
I20260811 12:46:49.639730 13858 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:49 dist-test-slave-f0tn krb5kdc[13840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452409, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:49.642751 13858 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:46:49.644102 13858 webserver.cc:460] Webserver started at http://127.7.111.126:40089/ using document root <none> and password file <none>
I20260811 12:46:49.644748 13858 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:49.644862 13858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:49.645169 13858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:49.648102 13858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "6797a61b8b4447dd906c5609b28f5160"
format_stamp: "Formatted at 2026-08-11 12:46:49 on dist-test-slave-f0tn"
server_key: "fc9289528654bcbe2ccb2940528503c9"
server_key_iv: "9e7e7d948e2790500591a74341d41e40"
server_key_version: "encryptionkey@0"
I20260811 12:46:49.648867 13858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "6797a61b8b4447dd906c5609b28f5160"
format_stamp: "Formatted at 2026-08-11 12:46:49 on dist-test-slave-f0tn"
server_key: "fc9289528654bcbe2ccb2940528503c9"
server_key_iv: "9e7e7d948e2790500591a74341d41e40"
server_key_version: "encryptionkey@0"
I20260811 12:46:49.653717 13858 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 12:46:49.657022 13883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:49.658318 13858 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:46:49.658478 13858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "6797a61b8b4447dd906c5609b28f5160"
format_stamp: "Formatted at 2026-08-11 12:46:49 on dist-test-slave-f0tn"
server_key: "fc9289528654bcbe2ccb2940528503c9"
server_key_iv: "9e7e7d948e2790500591a74341d41e40"
server_key_version: "encryptionkey@0"
I20260811 12:46:49.658653 13858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:49.670759 13858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:49.674765 13858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:49.675015 13858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:49.692202 13858 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:35367
I20260811 12:46:49.692233 13971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:35367 every 8 connection(s)
I20260811 12:46:49.693543 13858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:46:49.697116 13973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:49.698197 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 13858
I20260811 12:46:49.698433 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:46:49.699260 7613 external_mini_cluster.cc:1428] Setting key d6b8a378ac7e969406e1036a78af29e3
I20260811 12:46:49.709847 13973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6797a61b8b4447dd906c5609b28f5160: Bootstrap starting.
Aug 11 12:46:49 dist-test-slave-f0tn krb5kdc[13840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:49.713002 13973 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6797a61b8b4447dd906c5609b28f5160: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:49.714210 13973 log.cc:824] T 00000000000000000000000000000000 P 6797a61b8b4447dd906c5609b28f5160: Log is configured to *not* fsync() on all Append() calls
I20260811 12:46:49.717543 13973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6797a61b8b4447dd906c5609b28f5160: No bootstrap required, opened a new log
I20260811 12:46:49.721576 13979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:49.701366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41656 (local address 127.7.111.126:35367)
0811 12:46:49.702189 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:49.702201 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:49.702242 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:46:49.703853 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:49.703875 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:49.704233 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:49.704560 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:49.706021 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:49.707358 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:49.708611 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:49.709248 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:49.709442 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:49.713591 (+ 4149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:49.713598 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:49.713612 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:49.713646 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:49.717327 (+ 3681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:49.718012 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:49.718020 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:49.718028 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:49.718231 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:49.718538 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:49.718542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:49.718545 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:49.719228 (+ 683us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:49.719542 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:49.719840 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 12:46:49.720267 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":367,"threads_started":1}
I20260811 12:46:49.726970 13973 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6797a61b8b4447dd906c5609b28f5160 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6797a61b8b4447dd906c5609b28f5160" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 35367 } }
I20260811 12:46:49.727289 13973 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6797a61b8b4447dd906c5609b28f5160 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:49.727358 13973 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6797a61b8b4447dd906c5609b28f5160 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6797a61b8b4447dd906c5609b28f5160, State: Initialized, Role: FOLLOWER
I20260811 12:46:49.727805 13973 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6797a61b8b4447dd906c5609b28f5160 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6797a61b8b4447dd906c5609b28f5160" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 35367 } }
I20260811 12:46:49.728092 13973 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6797a61b8b4447dd906c5609b28f5160 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:46:49.728266 13973 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6797a61b8b4447dd906c5609b28f5160 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:46:49.728406 13973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6797a61b8b4447dd906c5609b28f5160 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:49.730835 13973 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6797a61b8b4447dd906c5609b28f5160 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6797a61b8b4447dd906c5609b28f5160" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 35367 } }
I20260811 12:46:49.731308 13973 leader_election.cc:302] T 00000000000000000000000000000000 P 6797a61b8b4447dd906c5609b28f5160 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6797a61b8b4447dd906c5609b28f5160; no voters:
I20260811 12:46:49.731850 13973 leader_election.cc:288] T 00000000000000000000000000000000 P 6797a61b8b4447dd906c5609b28f5160 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:46:49.731963 13982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6797a61b8b4447dd906c5609b28f5160 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:46:49.732594 13982 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6797a61b8b4447dd906c5609b28f5160 [term 1 LEADER]: Becoming Leader. State: Replica: 6797a61b8b4447dd906c5609b28f5160, State: Running, Role: LEADER
I20260811 12:46:49.733098 13982 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6797a61b8b4447dd906c5609b28f5160 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6797a61b8b4447dd906c5609b28f5160" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 35367 } }
I20260811 12:46:49.733796 13973 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6797a61b8b4447dd906c5609b28f5160 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:46:49.735965 13984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6797a61b8b4447dd906c5609b28f5160 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6797a61b8b4447dd906c5609b28f5160" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6797a61b8b4447dd906c5609b28f5160" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 35367 } } }
I20260811 12:46:49.736301 13984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6797a61b8b4447dd906c5609b28f5160 [sys.catalog]: This master's current role is: LEADER
I20260811 12:46:49.737108 13990 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:46:49.736959 13985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6797a61b8b4447dd906c5609b28f5160 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6797a61b8b4447dd906c5609b28f5160. Latest consensus state: current_term: 1 leader_uuid: "6797a61b8b4447dd906c5609b28f5160" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6797a61b8b4447dd906c5609b28f5160" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 35367 } } }
I20260811 12:46:49.738395 13985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6797a61b8b4447dd906c5609b28f5160 [sys.catalog]: This master's current role is: LEADER
I20260811 12:46:49.742889 13990 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:46:49.750602 13990 catalog_manager.cc:1347] Generated new cluster ID: 253a3c9bcdaa413f9bbfff07310297ac
I20260811 12:46:49.750712 13990 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:46:49.759482 13990 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:46:49.761279 13990 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:46:49.777879 13990 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6797a61b8b4447dd906c5609b28f5160: Generated new TSK 0
I20260811 12:46:49.779058 13990 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.111.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
I20260811 12:46:49.836894 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:35367
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:43275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:50.169399 14020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:50.169696 14020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:50.169802 14020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:50.176786 14020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:50.176880 14020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:50.177006 14020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:46:50.186683 14020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:43275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:35367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:50.187968 14020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:50.189232 14020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:50.201822 14026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:50.201792 14030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:50.201862 14025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:50.202971 14020 server_base.cc:1034] running on GCE node
I20260811 12:46:50.203503 14020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:50.204335 14020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:50.205521 14020 hybrid_clock.cc:648] HybridClock initialized: now 1786452410205474 us; error 34 us; skew 500 ppm
I20260811 12:46:50.205823 14020 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:50 dist-test-slave-f0tn krb5kdc[13840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452410, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:50.208841 14020 init.cc:377] Logged in from keytab as kudu/127.7.111.65@KRBTEST.COM (short username kudu)
I20260811 12:46:50.210295 14020 webserver.cc:460] Webserver started at http://127.7.111.65:44847/ using document root <none> and password file <none>
I20260811 12:46:50.210981 14020 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:50.211095 14020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:50.211396 14020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:50.214329 14020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "6d528c7816f74f75bd7edada2e21664f"
format_stamp: "Formatted at 2026-08-11 12:46:50 on dist-test-slave-f0tn"
server_key: "2af0c6b49a9145cac560ee3166c94327"
server_key_iv: "e85fbbe4e1a2fd455eeba4de95c84028"
server_key_version: "encryptionkey@0"
I20260811 12:46:50.215066 14020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "6d528c7816f74f75bd7edada2e21664f"
format_stamp: "Formatted at 2026-08-11 12:46:50 on dist-test-slave-f0tn"
server_key: "2af0c6b49a9145cac560ee3166c94327"
server_key_iv: "e85fbbe4e1a2fd455eeba4de95c84028"
server_key_version: "encryptionkey@0"
I20260811 12:46:50.219975 14020 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:46:50.223353 14044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:50.224606 14020 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:46:50.224754 14020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "6d528c7816f74f75bd7edada2e21664f"
format_stamp: "Formatted at 2026-08-11 12:46:50 on dist-test-slave-f0tn"
server_key: "2af0c6b49a9145cac560ee3166c94327"
server_key_iv: "e85fbbe4e1a2fd455eeba4de95c84028"
server_key_version: "encryptionkey@0"
I20260811 12:46:50.224938 14020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:50.241905 14020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:50.245818 14020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:50.246070 14020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:50.247048 14020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:46:50.248349 14020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:50.248427 14020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:50.248548 14020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:50.248621 14020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:50.279263 14020 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:45003
I20260811 12:46:50.279335 14235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:45003 every 8 connection(s)
I20260811 12:46:50.280573 14020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:46:50.289165 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 14020
I20260811 12:46:50.289403 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:46:50.290068 7613 external_mini_cluster.cc:1428] Setting key 00daec9eb0bb6fe0ef4ac41b4ce3690d
Aug 11 12:46:50 dist-test-slave-f0tn krb5kdc[13840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452410, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:50.300525 14238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:50.283115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:33393 (local address 127.7.111.126:35367)
0811 12:46:50.283525 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:50.283530 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:50.284531 (+ 1001us) server_negotiation.cc:407] Connection header received
0811 12:46:50.286383 (+ 1852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:50.286388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:50.286459 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:50.286559 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:50.287948 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:50.288665 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:50.290261 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:50.290824 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:50.290925 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:50.294706 (+ 3781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:50.294713 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:50.294717 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:50.294755 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:50.297667 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:50.298337 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:50.298342 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:50.298346 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:50.298485 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:50.298870 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:50.298874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:50.298877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:50.299188 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:50.299329 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:50.299980 (+ 651us) server_negotiation.cc:299] Negotiation successful
0811 12:46:50.300184 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":195,"threads_started":1}
I20260811 12:46:50.301430 14239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:50.283693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:35367 (local address 127.7.111.65:33393)
0811 12:46:50.284370 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:50.284410 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:50.286128 (+ 1718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:50.286760 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:50.286768 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:50.287254 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:50.287818 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:50.287836 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:50.288792 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:50.288795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:50.290113 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:50.290126 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:50.291225 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:50.291233 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:50.291514 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:50.292207 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:50.292237 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:50.294487 (+ 2250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:50.297795 (+ 3308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:50.297802 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:50.297814 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:50.298215 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:50.298577 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:50.298580 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:50.298583 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:50.298768 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:50.299302 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:50.299307 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:50.299611 (+ 304us) client_negotiation.cc:769] Sending connection context
0811 12:46:50.299850 (+ 239us) client_negotiation.cc:240] Negotiation successful
0811 12:46:50.300212 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":256,"threads_started":1}
I20260811 12:46:50.302955 14236 heartbeater.cc:347] Connected to a master server at 127.7.111.126:35367
I20260811 12:46:50.303278 14236 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:50.304126 14236 heartbeater.cc:511] Master 127.7.111.126:35367 requested a full tablet report, sending...
I20260811 12:46:50.306593 13912 ts_manager.cc:194] Registered new tserver with Master: 6d528c7816f74f75bd7edada2e21664f (127.7.111.65:45003)
I20260811 12:46:50.308498 13912 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:33393
WARNING: no policy specified for kudu/127.7.111.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
I20260811 12:46:50.343950 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:35367
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:43275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:50.679596 14244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:50.679867 14244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:50.680013 14244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:50.686986 14244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:50.687095 14244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:50.687265 14244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:46:50.696671 14244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:43275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:35367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:50.697888 14244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:50.699143 14244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:50.711583 14259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:50.711656 14262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:50.711627 14260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:50.712693 14244 server_base.cc:1034] running on GCE node
I20260811 12:46:50.713284 14244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:50.714221 14244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:50.715415 14244 hybrid_clock.cc:648] HybridClock initialized: now 1786452410715374 us; error 32 us; skew 500 ppm
I20260811 12:46:50.715713 14244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:50 dist-test-slave-f0tn krb5kdc[13840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452410, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:50.718708 14244 init.cc:377] Logged in from keytab as kudu/127.7.111.66@KRBTEST.COM (short username kudu)
I20260811 12:46:50.720031 14244 webserver.cc:460] Webserver started at http://127.7.111.66:38545/ using document root <none> and password file <none>
I20260811 12:46:50.720703 14244 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:50.720804 14244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:50.721053 14244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:50.723914 14244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "b63535d2c4d44932984fc31ec9bbad0a"
format_stamp: "Formatted at 2026-08-11 12:46:50 on dist-test-slave-f0tn"
server_key: "3fed7d5c8fd493cba5192600b9252747"
server_key_iv: "e57ec181cdfe654cda06d3f46e83f757"
server_key_version: "encryptionkey@0"
I20260811 12:46:50.724627 14244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "b63535d2c4d44932984fc31ec9bbad0a"
format_stamp: "Formatted at 2026-08-11 12:46:50 on dist-test-slave-f0tn"
server_key: "3fed7d5c8fd493cba5192600b9252747"
server_key_iv: "e57ec181cdfe654cda06d3f46e83f757"
server_key_version: "encryptionkey@0"
I20260811 12:46:50.729576 14244 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:46:50.732906 14276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:50.734251 14244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:46:50.734401 14244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "b63535d2c4d44932984fc31ec9bbad0a"
format_stamp: "Formatted at 2026-08-11 12:46:50 on dist-test-slave-f0tn"
server_key: "3fed7d5c8fd493cba5192600b9252747"
server_key_iv: "e57ec181cdfe654cda06d3f46e83f757"
server_key_version: "encryptionkey@0"
I20260811 12:46:50.734586 14244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:50.764997 14244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:50.769282 14244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:50.769569 14244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:50.770563 14244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:46:50.771906 14244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:50.771979 14244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:50.772070 14244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:50.772121 14244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:50.802584 14244 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:37601
I20260811 12:46:50.802698 14460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:37601 every 8 connection(s)
I20260811 12:46:50.803958 14244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:46:50.804497 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 14244
I20260811 12:46:50.804719 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:46:50.805279 7613 external_mini_cluster.cc:1428] Setting key 15c75776a5feb9e18f330c2a930f0d6d
Aug 11 12:46:50 dist-test-slave-f0tn krb5kdc[13840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452410, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:50.824533 14464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:50.806694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:48585 (local address 127.7.111.126:35367)
0811 12:46:50.807118 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:50.807124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:50.808484 (+ 1360us) server_negotiation.cc:407] Connection header received
0811 12:46:50.810442 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:50.810447 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:50.810527 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:50.810645 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:50.812114 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:50.813054 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:50.814630 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:50.815210 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:50.815329 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:50.818944 (+ 3615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:50.818949 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:50.818953 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:50.818985 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:50.821498 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:50.822304 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:50.822307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:50.822310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:50.822429 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:50.822915 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:50.822918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:50.822920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:50.823163 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:50.823268 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:50.824093 (+ 825us) server_negotiation.cc:299] Negotiation successful
0811 12:46:50.824259 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":228,"threads_started":1}
I20260811 12:46:50.825963 14466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:50.807441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:35367 (local address 127.7.111.66:48585)
0811 12:46:50.808306 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:50.808353 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:50.810178 (+ 1825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:50.810828 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:50.810836 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:50.811387 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:50.811941 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:50.811958 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:50.813203 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:50.813207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:50.814493 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:50.814505 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:50.815397 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:50.815402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:50.815596 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:50.816222 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:50.816244 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:50.818719 (+ 2475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:50.821645 (+ 2926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:50.821653 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:50.821668 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:50.822205 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:50.822554 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:50.822559 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:50.822562 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:50.822822 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:50.823277 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:50.823284 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:50.823692 (+ 408us) client_negotiation.cc:769] Sending connection context
0811 12:46:50.824043 (+ 351us) client_negotiation.cc:240] Negotiation successful
0811 12:46:50.824559 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":349,"threads_started":1}
I20260811 12:46:50.827409 14461 heartbeater.cc:347] Connected to a master server at 127.7.111.126:35367
I20260811 12:46:50.827766 14461 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:50.828722 14461 heartbeater.cc:511] Master 127.7.111.126:35367 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.111.67@KRBTEST.COM; defaulting to no policy
I20260811 12:46:50.830374 13912 ts_manager.cc:194] Registered new tserver with Master: b63535d2c4d44932984fc31ec9bbad0a (127.7.111.66:37601)
I20260811 12:46:50.831164 13912 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:48585
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
I20260811 12:46:50.856058 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:35367
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:43275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:51.178115 14471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:51.178396 14471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:51.178535 14471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:51.185659 14471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:51.185772 14471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:51.185923 14471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:46:51.195310 14471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:43275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:35367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:51.196557 14471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:51.197906 14471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:51.210563 14477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:51.210664 14480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:51.210639 14483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:51.211472 14471 server_base.cc:1034] running on GCE node
I20260811 12:46:51.212071 14471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:51.212997 14471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:51.214210 14471 hybrid_clock.cc:648] HybridClock initialized: now 1786452411214151 us; error 44 us; skew 500 ppm
I20260811 12:46:51.214556 14471 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:51 dist-test-slave-f0tn krb5kdc[13840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452411, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:51.217638 14471 init.cc:377] Logged in from keytab as kudu/127.7.111.67@KRBTEST.COM (short username kudu)
I20260811 12:46:51.219033 14471 webserver.cc:460] Webserver started at http://127.7.111.67:40441/ using document root <none> and password file <none>
I20260811 12:46:51.219678 14471 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:51.219810 14471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:51.220109 14471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:51.223073 14471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "432b600f6171446e95b140497b2942ba"
format_stamp: "Formatted at 2026-08-11 12:46:51 on dist-test-slave-f0tn"
server_key: "9064de34a112f6384cb5a741bfa12625"
server_key_iv: "f78adb2ddc9026b06fc3e6a0defe32ae"
server_key_version: "encryptionkey@0"
I20260811 12:46:51.223839 14471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "432b600f6171446e95b140497b2942ba"
format_stamp: "Formatted at 2026-08-11 12:46:51 on dist-test-slave-f0tn"
server_key: "9064de34a112f6384cb5a741bfa12625"
server_key_iv: "f78adb2ddc9026b06fc3e6a0defe32ae"
server_key_version: "encryptionkey@0"
I20260811 12:46:51.228899 14471 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:46:51.232323 14500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:51.233939 14471 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:46:51.234076 14471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "432b600f6171446e95b140497b2942ba"
format_stamp: "Formatted at 2026-08-11 12:46:51 on dist-test-slave-f0tn"
server_key: "9064de34a112f6384cb5a741bfa12625"
server_key_iv: "f78adb2ddc9026b06fc3e6a0defe32ae"
server_key_version: "encryptionkey@0"
I20260811 12:46:51.234256 14471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:51.256031 14471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:51.259773 14471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:51.260039 14471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:51.260980 14471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:46:51.262315 14471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:51.262405 14471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:51.262516 14471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:51.262566 14471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:51.293998 14471 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:33633
I20260811 12:46:51.294054 14694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:33633 every 8 connection(s)
I20260811 12:46:51.295276 14471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:46:51.296132 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 14471
I20260811 12:46:51.296352 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:46:51.296846 7613 external_mini_cluster.cc:1428] Setting key ba4ef41e8b38dc12669f8d6b958b0c0f
Aug 11 12:46:51 dist-test-slave-f0tn krb5kdc[13840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452411, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:51.312040 14236 heartbeater.cc:503] Master 127.7.111.126:35367 was elected leader, sending a full tablet report...
I20260811 12:46:51.315243 14464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:51.298060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:35643 (local address 127.7.111.126:35367)
0811 12:46:51.298252 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:51.298258 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:51.299572 (+ 1314us) server_negotiation.cc:407] Connection header received
0811 12:46:51.301578 (+ 2006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:51.301582 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:51.301650 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:51.301733 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:51.303239 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:51.303972 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:51.305486 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:51.305922 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:51.306017 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:51.309262 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:51.309267 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:51.309270 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:51.309303 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:51.312308 (+ 3005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:51.313121 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:51.313127 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:51.313131 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:51.313255 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:51.313728 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:51.313732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:51.313735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:51.314034 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:51.314160 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:51.314824 (+ 664us) server_negotiation.cc:299] Negotiation successful
0811 12:46:51.315006 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:46:51.316182 14698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:51.298535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:35367 (local address 127.7.111.67:35643)
0811 12:46:51.299388 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:51.299452 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:51.301302 (+ 1850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:51.301922 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:51.301929 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:51.302446 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:51.303077 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:51.303098 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:51.304089 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:51.304093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:51.305319 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:51.305332 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:51.306101 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:51.306106 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:51.306314 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:51.306892 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:51.306915 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:51.309057 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:51.312469 (+ 3412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:51.312476 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:51.312489 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:51.313010 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:51.313362 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:51.313367 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:51.313369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:51.313592 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:51.314146 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:51.314153 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:51.314486 (+ 333us) client_negotiation.cc:769] Sending connection context
0811 12:46:51.314715 (+ 229us) client_negotiation.cc:240] Negotiation successful
0811 12:46:51.315115 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":317,"threads_started":1}
I20260811 12:46:51.317534 14696 heartbeater.cc:347] Connected to a master server at 127.7.111.126:35367
I20260811 12:46:51.317824 14696 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:51.318552 14696 heartbeater.cc:511] Master 127.7.111.126:35367 requested a full tablet report, sending...
I20260811 12:46:51.319954 13912 ts_manager.cc:194] Registered new tserver with Master: 432b600f6171446e95b140497b2942ba (127.7.111.67:33633)
I20260811 12:46:51.320766 13912 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:35643
I20260811 12:46:51.324926 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 12:46:51 dist-test-slave-f0tn krb5kdc[13840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 12:46:51 dist-test-slave-f0tn krb5kdc[13840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.65@KRBTEST.COM
I20260811 12:46:51.380499 14705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:51.365893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41694 (local address 127.7.111.65:45003)
0811 12:46:51.366555 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:51.366563 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:51.366592 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:46:51.366658 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:51.366665 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:51.366892 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:51.367024 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:51.368346 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:51.369419 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:51.370605 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:51.371195 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:51.371296 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:51.375087 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:51.375102 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:51.375119 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:51.375158 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:51.378195 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:51.378796 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:51.378803 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:51.378806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:51.378927 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:51.379198 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:51.379201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:51.379203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:51.379546 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:51.379785 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:51.379995 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 12:46:51.380201 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":352,"threads_started":1}
Aug 11 12:46:51 dist-test-slave-f0tn krb5kdc[13840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:51.398458 14464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:51.384948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41658 (local address 127.7.111.126:35367)
0811 12:46:51.385157 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:51.385162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:51.385331 (+ 169us) server_negotiation.cc:407] Connection header received
0811 12:46:51.385538 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:51.385544 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:51.385627 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:51.385719 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:51.386930 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:51.387835 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:51.389118 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:51.389588 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:51.389703 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:51.393443 (+ 3740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:51.393450 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:51.393455 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:51.393495 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:51.395871 (+ 2376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:51.396494 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:51.396500 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:51.396503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:51.396626 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:51.396990 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:51.396995 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:51.396997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:51.397326 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:51.397589 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:51.397906 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 12:46:51.398122 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 12:46:51.399049 13912 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41658
I20260811 12:46:51.400010 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 14020
I20260811 12:46:51.412155 14705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:51.400830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41696 (local address 127.7.111.65:45003)
0811 12:46:51.400979 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:51.400984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:51.401274 (+ 290us) server_negotiation.cc:407] Connection header received
0811 12:46:51.401426 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:51.401429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:51.401492 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:51.401569 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:51.402772 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:51.403516 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:51.404721 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:51.405313 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:51.405483 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:51.407578 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:51.407583 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:51.407586 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:51.407618 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:51.410165 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:51.410728 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:51.410732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:51.410734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:51.410850 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:51.411124 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:51.411126 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:51.411128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:51.411357 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:51.411522 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:51.411761 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 12:46:51.411925 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 12:46:51.412744 14230 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:51.496824 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 14020
I20260811 12:46:51.522359 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 14244
Aug 11 12:46:51 dist-test-slave-f0tn krb5kdc[13840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.66@KRBTEST.COM
I20260811 12:46:51.542299 14712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:51.523345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39918 (local address 127.7.111.66:37601)
0811 12:46:51.523894 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:51.523901 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:51.523927 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:46:51.523995 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:51.524003 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:51.524219 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:51.524350 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:51.525559 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:51.526576 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:51.527848 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:51.528397 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:51.528499 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:51.536750 (+ 8251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:51.536763 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:51.536780 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:51.536816 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:51.539889 (+ 3073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:51.540487 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:51.540494 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:51.540496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:51.540617 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:51.540922 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:51.540926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:51.540928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:51.541299 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:51.541558 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:51.541785 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 12:46:51.541978 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":277,"threads_started":1}
I20260811 12:46:51.543529 14453 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:51.627288 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 14244
I20260811 12:46:51.653188 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 14471
Aug 11 12:46:51 dist-test-slave-f0tn krb5kdc[13840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.67@KRBTEST.COM
I20260811 12:46:51.670827 14715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:51.654353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37128 (local address 127.7.111.67:33633)
0811 12:46:51.654991 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:51.654998 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:51.655029 (+ 31us) server_negotiation.cc:407] Connection header received
0811 12:46:51.655094 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:51.655101 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:51.655348 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:51.655484 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:51.656883 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:51.658057 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:51.659450 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:51.660139 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:51.660260 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:51.664818 (+ 4558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:51.664836 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:51.664854 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:51.664905 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:51.667925 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:51.668717 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:51.668726 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:51.668728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:51.668879 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:51.669216 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:51.669220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:51.669222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:51.669717 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:51.669996 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:51.670280 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 12:46:51.670509 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":322,"threads_started":1}
I20260811 12:46:51.672194 14689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:51.754640 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 14471
I20260811 12:46:51.779377 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 13858
I20260811 12:46:51.780618 13964 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:51.884639 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 13858
2026-08-11T12:46:51Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4053 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:46:51 dist-test-slave-f0tn krb5kdc[14726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:46:51 dist-test-slave-f0tn krb5kdc[14726](info): set up 2 sockets
Aug 11 12:46:51 dist-test-slave-f0tn krb5kdc[14726](info): commencing operation
krb5kdc: starting...
W20260811 12:46:53.123095 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.001s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:46:53 dist-test-slave-f0tn krb5kdc[14726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:46:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:46:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:46:53.348572 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:37065
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:37065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:53.670365 14751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:53.670683 14751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:53.670789 14751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:53.678155 14751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:46:53.678282 14751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:53.678335 14751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:53.678395 14751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:46:53.678442 14751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:46:53.688630 14751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:37065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:37065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:53.690011 14751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:53.691339 14751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:53.702543 14758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:53.702502 14761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:53.702790 14765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:53.703306 14751 server_base.cc:1034] running on GCE node
I20260811 12:46:53.704062 14751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:53.705471 14751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:53.706691 14751 hybrid_clock.cc:648] HybridClock initialized: now 1786452413706620 us; error 60 us; skew 500 ppm
I20260811 12:46:53.707039 14751 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:53 dist-test-slave-f0tn krb5kdc[14726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452413, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:53.710397 14751 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:46:53.711522 14751 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:53.711637 14751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:53.711951 14751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:53.715036 14751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "39b1e42232754517be52045a3df5e428"
format_stamp: "Formatted at 2026-08-11 12:46:53 on dist-test-slave-f0tn"
server_key: "8a6178997d6da5b484613cc40a0e4076"
server_key_iv: "a5c9cbd5ae97f11e06cf8a478fe9c020"
server_key_version: "encryptionkey@0"
I20260811 12:46:53.715826 14751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "39b1e42232754517be52045a3df5e428"
format_stamp: "Formatted at 2026-08-11 12:46:53 on dist-test-slave-f0tn"
server_key: "8a6178997d6da5b484613cc40a0e4076"
server_key_iv: "a5c9cbd5ae97f11e06cf8a478fe9c020"
server_key_version: "encryptionkey@0"
I20260811 12:46:53.720963 14751 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:46:53.724609 14774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:53.726002 14751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:46:53.726202 14751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "39b1e42232754517be52045a3df5e428"
format_stamp: "Formatted at 2026-08-11 12:46:53 on dist-test-slave-f0tn"
server_key: "8a6178997d6da5b484613cc40a0e4076"
server_key_iv: "a5c9cbd5ae97f11e06cf8a478fe9c020"
server_key_version: "encryptionkey@0"
I20260811 12:46:53.726444 14751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:53.739259 14751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:53.743140 14751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:53.743398 14751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:53.760805 14751 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:37065
I20260811 12:46:53.760842 14864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:37065 every 8 connection(s)
I20260811 12:46:53.761767 14751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:46:53.765478 14866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:53.767315 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 14751
I20260811 12:46:53.767563 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:46:53.768353 7613 external_mini_cluster.cc:1428] Setting key a04b52b357478f9eae4b16ee20246a5c
I20260811 12:46:53.777812 14866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428: Bootstrap starting.
I20260811 12:46:53.781569 14866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:53.782863 14866 log.cc:824] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428: Log is configured to *not* fsync() on all Append() calls
Aug 11 12:46:53 dist-test-slave-f0tn krb5kdc[14726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:53.785852 14866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428: No bootstrap required, opened a new log
I20260811 12:46:53.791584 14870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:53.770503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51976 (local address 127.7.111.126:37065)
0811 12:46:53.771308 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:53.771320 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:53.771366 (+ 46us) server_negotiation.cc:407] Connection header received
0811 12:46:53.773044 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:53.773070 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:53.773498 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:53.773821 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:53.775646 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:53.776928 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:53.778682 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:53.779365 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:53.779523 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:53.784384 (+ 4861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:53.784392 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:53.784407 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:53.784442 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:53.787359 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:53.788059 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:53.788066 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:53.788072 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:53.788252 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:53.788655 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:53.788659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:53.788661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:53.789266 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:53.789584 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:53.790102 (+ 518us) server_negotiation.cc:299] Negotiation successful
0811 12:46:53.790514 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":369,"threads_started":1}
I20260811 12:46:53.793557 14866 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39b1e42232754517be52045a3df5e428" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37065 } }
I20260811 12:46:53.793902 14866 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:53.793994 14866 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39b1e42232754517be52045a3df5e428, State: Initialized, Role: FOLLOWER
I20260811 12:46:53.794569 14866 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39b1e42232754517be52045a3df5e428" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37065 } }
I20260811 12:46:53.794790 14866 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:46:53.794914 14866 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:46:53.795063 14866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:53.797223 14866 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39b1e42232754517be52045a3df5e428" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37065 } }
I20260811 12:46:53.797761 14866 leader_election.cc:302] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39b1e42232754517be52045a3df5e428; no voters:
I20260811 12:46:53.798336 14866 leader_election.cc:288] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:46:53.798475 14874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:46:53.799134 14874 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428 [term 1 LEADER]: Becoming Leader. State: Replica: 39b1e42232754517be52045a3df5e428, State: Running, Role: LEADER
I20260811 12:46:53.799710 14874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39b1e42232754517be52045a3df5e428" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37065 } }
I20260811 12:46:53.800377 14866 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:46:53.802362 14875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39b1e42232754517be52045a3df5e428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39b1e42232754517be52045a3df5e428" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37065 } } }
I20260811 12:46:53.802796 14875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428 [sys.catalog]: This master's current role is: LEADER
I20260811 12:46:53.802721 14879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39b1e42232754517be52045a3df5e428. Latest consensus state: current_term: 1 leader_uuid: "39b1e42232754517be52045a3df5e428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39b1e42232754517be52045a3df5e428" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37065 } } }
I20260811 12:46:53.803059 14879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428 [sys.catalog]: This master's current role is: LEADER
I20260811 12:46:53.804418 14890 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:46:53.810123 14890 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:46:53.820204 14890 catalog_manager.cc:1347] Generated new cluster ID: 9a51dc5fcbcc4f8b8eec313a6a0d9c65
I20260811 12:46:53.820326 14890 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:46:53.841449 14890 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:46:53.842772 14890 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:46:53.850121 14890 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428: Generated new TSK 0
I20260811 12:46:53.850885 14890 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.111.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
I20260811 12:46:53.905097 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:37065
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:54.237334 14919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:54.237690 14919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:54.237838 14919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:54.245229 14919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:54.245332 14919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:54.245512 14919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:46:54.255352 14919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:37065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:54.256608 14919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:54.257959 14919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:54.270586 14933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:54.270565 14931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:54.270819 14930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:54.271270 14919 server_base.cc:1034] running on GCE node
I20260811 12:46:54.271888 14919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:54.272815 14919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:54.274050 14919 hybrid_clock.cc:648] HybridClock initialized: now 1786452414273993 us; error 46 us; skew 500 ppm
I20260811 12:46:54.274391 14919 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:54 dist-test-slave-f0tn krb5kdc[14726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452414, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:54.277601 14919 init.cc:377] Logged in from keytab as kudu/127.7.111.65@KRBTEST.COM (short username kudu)
I20260811 12:46:54.278759 14919 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:54.278870 14919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:54.279196 14919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:54.282226 14919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "a78d91862daa43be89d40d792aa9c020"
format_stamp: "Formatted at 2026-08-11 12:46:54 on dist-test-slave-f0tn"
server_key: "c9a70a46e70bec2204226560d51fd10d"
server_key_iv: "9152f41161d93be66679a78ee88a6c29"
server_key_version: "encryptionkey@0"
I20260811 12:46:54.283030 14919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "a78d91862daa43be89d40d792aa9c020"
format_stamp: "Formatted at 2026-08-11 12:46:54 on dist-test-slave-f0tn"
server_key: "c9a70a46e70bec2204226560d51fd10d"
server_key_iv: "9152f41161d93be66679a78ee88a6c29"
server_key_version: "encryptionkey@0"
I20260811 12:46:54.288515 14919 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:46:54.292151 14941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:54.293627 14919 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:46:54.293802 14919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "a78d91862daa43be89d40d792aa9c020"
format_stamp: "Formatted at 2026-08-11 12:46:54 on dist-test-slave-f0tn"
server_key: "c9a70a46e70bec2204226560d51fd10d"
server_key_iv: "9152f41161d93be66679a78ee88a6c29"
server_key_version: "encryptionkey@0"
I20260811 12:46:54.294059 14919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:54.314553 14919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:54.318553 14919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:54.318831 14919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:54.319828 14919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:46:54.321262 14919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:54.321372 14919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:54.321516 14919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:54.321599 14919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:54.353830 14919 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:40827
I20260811 12:46:54.353912 15136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:40827 every 8 connection(s)
I20260811 12:46:54.354665 14919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:46:54.356926 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 14919
I20260811 12:46:54.357156 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:46:54.357987 7613 external_mini_cluster.cc:1428] Setting key e38d206ccd21c6082e084f4aff35fb27
Aug 11 12:46:54 dist-test-slave-f0tn krb5kdc[14726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452414, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:54.375032 15139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:54.357556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:57005 (local address 127.7.111.126:37065)
0811 12:46:54.357985 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:54.357989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:54.359149 (+ 1160us) server_negotiation.cc:407] Connection header received
0811 12:46:54.361040 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:54.361046 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:54.361126 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:54.361232 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:54.362799 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:54.363529 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:54.365083 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:54.365563 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:54.365681 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:54.369257 (+ 3576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:54.369262 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:54.369265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:54.369296 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:54.372137 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:54.372814 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:54.372819 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:54.372822 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:54.372960 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:54.373338 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:54.373343 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:54.373345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:54.373683 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:54.373831 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:54.374486 (+ 655us) server_negotiation.cc:299] Negotiation successful
0811 12:46:54.374696 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":222,"threads_started":1}
I20260811 12:46:54.375909 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:54.357904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:37065 (local address 127.7.111.65:57005)
0811 12:46:54.358714 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:54.358766 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:54.360774 (+ 2008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:54.361513 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:54.361522 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:54.362059 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:54.362618 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:54.362637 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:54.363684 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:54.363689 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:54.364928 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:54.364942 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:54.365797 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:54.365805 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:54.366031 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:54.366657 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:54.366687 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:54.369029 (+ 2342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:54.372268 (+ 3239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:54.372274 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:54.372286 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:54.372697 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:54.373056 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:54.373059 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:54.373061 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:54.373238 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:54.373827 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:54.373833 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:54.374124 (+ 291us) client_negotiation.cc:769] Sending connection context
0811 12:46:54.374356 (+ 232us) client_negotiation.cc:240] Negotiation successful
0811 12:46:54.374738 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":320,"threads_started":1}
I20260811 12:46:54.377424 15137 heartbeater.cc:347] Connected to a master server at 127.7.111.126:37065
I20260811 12:46:54.377761 15137 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:54.378543 15137 heartbeater.cc:511] Master 127.7.111.126:37065 requested a full tablet report, sending...
I20260811 12:46:54.380749 14801 ts_manager.cc:194] Registered new tserver with Master: a78d91862daa43be89d40d792aa9c020 (127.7.111.65:40827)
WARNING: no policy specified for kudu/127.7.111.66@KRBTEST.COM; defaulting to no policy
I20260811 12:46:54.382270 14801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:57005
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
I20260811 12:46:54.410012 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:37065
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:54.745893 15143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:54.746165 15143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:54.746264 15143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:54.753304 15143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:54.753428 15143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:54.753593 15143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:46:54.764794 15143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:37065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:54.766312 15143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:54.767913 15143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:54.781802 15149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:54.781834 15150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:54.781860 15154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:54.782776 15143 server_base.cc:1034] running on GCE node
I20260811 12:46:54.783293 15143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:54.784126 15143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:54.785343 15143 hybrid_clock.cc:648] HybridClock initialized: now 1786452414785291 us; error 39 us; skew 500 ppm
I20260811 12:46:54.785676 15143 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:54 dist-test-slave-f0tn krb5kdc[14726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452414, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:54.788678 15143 init.cc:377] Logged in from keytab as kudu/127.7.111.66@KRBTEST.COM (short username kudu)
I20260811 12:46:54.789889 15143 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:54.789994 15143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:54.790298 15143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:54.793219 15143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "03b0a4d5eb2c46bd90f98d917384d677"
format_stamp: "Formatted at 2026-08-11 12:46:54 on dist-test-slave-f0tn"
server_key: "5174a78e539480ec9f097f3e53f16473"
server_key_iv: "fa312ab40a305efdfaedbb756cc529d8"
server_key_version: "encryptionkey@0"
I20260811 12:46:54.794075 15143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "03b0a4d5eb2c46bd90f98d917384d677"
format_stamp: "Formatted at 2026-08-11 12:46:54 on dist-test-slave-f0tn"
server_key: "5174a78e539480ec9f097f3e53f16473"
server_key_iv: "fa312ab40a305efdfaedbb756cc529d8"
server_key_version: "encryptionkey@0"
I20260811 12:46:54.798883 15143 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:46:54.802273 15163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:54.803483 15143 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:46:54.803645 15143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "03b0a4d5eb2c46bd90f98d917384d677"
format_stamp: "Formatted at 2026-08-11 12:46:54 on dist-test-slave-f0tn"
server_key: "5174a78e539480ec9f097f3e53f16473"
server_key_iv: "fa312ab40a305efdfaedbb756cc529d8"
server_key_version: "encryptionkey@0"
I20260811 12:46:54.803834 15143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:54.814747 15143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:54.818490 15143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:54.818782 15143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:54.819701 15143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:46:54.820967 15143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:54.821053 15143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:54.821130 15143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:54.821182 15143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:54.851428 15143 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:37853
I20260811 12:46:54.851482 15336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:37853 every 8 connection(s)
I20260811 12:46:54.852303 15143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:46:54.862564 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 15143
I20260811 12:46:54.862792 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:46:54.863481 7613 external_mini_cluster.cc:1428] Setting key 7b5e8da479beaac6b523551479db4e59
Aug 11 12:46:54 dist-test-slave-f0tn krb5kdc[14726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452414, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:54.871833 15139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:54.854718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:33395 (local address 127.7.111.126:37065)
0811 12:46:54.854884 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:54.854888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:54.856081 (+ 1193us) server_negotiation.cc:407] Connection header received
0811 12:46:54.857879 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:54.857884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:54.857956 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:54.858058 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:54.859643 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:54.860334 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:54.862086 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:54.862530 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:54.862622 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:54.866398 (+ 3776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:54.866403 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:54.866406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:54.866437 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:54.868865 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:54.869621 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:54.869624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:54.869627 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:54.869766 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:54.870186 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:54.870190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:54.870192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:54.870444 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:54.870575 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:54.871258 (+ 683us) server_negotiation.cc:299] Negotiation successful
0811 12:46:54.871484 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:46:54.872720 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:54.855108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:37065 (local address 127.7.111.66:33395)
0811 12:46:54.855890 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:54.855929 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:54.857639 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:54.858335 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:54.858344 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:54.858889 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:54.859483 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:54.859506 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:54.860505 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:54.860511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:54.861958 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:54.861973 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:54.862775 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:54.862784 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:54.863087 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:54.863812 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:54.863836 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:54.866188 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:54.868996 (+ 2808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:54.869004 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:54.869020 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:54.869504 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:54.869875 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:54.869880 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:54.869884 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:54.870069 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:54.870568 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:54.870575 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:54.870900 (+ 325us) client_negotiation.cc:769] Sending connection context
0811 12:46:54.871128 (+ 228us) client_negotiation.cc:240] Negotiation successful
0811 12:46:54.871625 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":333,"threads_started":1}
I20260811 12:46:54.874097 15337 heartbeater.cc:347] Connected to a master server at 127.7.111.126:37065
I20260811 12:46:54.874378 15337 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:54.874984 15337 heartbeater.cc:511] Master 127.7.111.126:37065 requested a full tablet report, sending...
I20260811 12:46:54.876412 14801 ts_manager.cc:194] Registered new tserver with Master: 03b0a4d5eb2c46bd90f98d917384d677 (127.7.111.66:37853)
I20260811 12:46:54.877197 14801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:33395
WARNING: no policy specified for kudu/127.7.111.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
I20260811 12:46:54.914149 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:37065
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:46:55.248878 15345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:55.249168 15345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:55.249269 15345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:55.256709 15345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:55.256824 15345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:55.256949 15345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:46:55.267050 15345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:37065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:55.268379 15345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:55.269770 15345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:55.282924 15351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:55.283056 15355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:55.282968 15353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:55.284003 15345 server_base.cc:1034] running on GCE node
I20260811 12:46:55.284651 15345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:55.285646 15345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:55.286854 15345 hybrid_clock.cc:648] HybridClock initialized: now 1786452415286802 us; error 43 us; skew 500 ppm
I20260811 12:46:55.287175 15345 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:46:55 dist-test-slave-f0tn krb5kdc[14726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452415, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:46:55.290185 15345 init.cc:377] Logged in from keytab as kudu/127.7.111.67@KRBTEST.COM (short username kudu)
I20260811 12:46:55.291287 15345 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:55.291392 15345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:55.291716 15345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:55.294706 15345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "9d1c5485bfb24d15b5bc95dd499ba9a0"
format_stamp: "Formatted at 2026-08-11 12:46:55 on dist-test-slave-f0tn"
server_key: "3389c4c53d073f19cb1f12a459e77810"
server_key_iv: "93f43b785cc4ada61d507e34df6939c6"
server_key_version: "encryptionkey@0"
I20260811 12:46:55.295508 15345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "9d1c5485bfb24d15b5bc95dd499ba9a0"
format_stamp: "Formatted at 2026-08-11 12:46:55 on dist-test-slave-f0tn"
server_key: "3389c4c53d073f19cb1f12a459e77810"
server_key_iv: "93f43b785cc4ada61d507e34df6939c6"
server_key_version: "encryptionkey@0"
I20260811 12:46:55.300635 15345 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:46:55.304371 15365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:55.305757 15345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:46:55.305907 15345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "9d1c5485bfb24d15b5bc95dd499ba9a0"
format_stamp: "Formatted at 2026-08-11 12:46:55 on dist-test-slave-f0tn"
server_key: "3389c4c53d073f19cb1f12a459e77810"
server_key_iv: "93f43b785cc4ada61d507e34df6939c6"
server_key_version: "encryptionkey@0"
I20260811 12:46:55.306118 15345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:55.321918 15345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:55.325881 15345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:55.326154 15345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:55.327147 15345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:46:55.328508 15345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:55.328615 15345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:55.328727 15345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:55.328811 15345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:55.360822 15345 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:37947
I20260811 12:46:55.360879 15541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:37947 every 8 connection(s)
I20260811 12:46:55.361696 15345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:46:55.368062 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 15345
I20260811 12:46:55.368257 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:46:55.368800 7613 external_mini_cluster.cc:1428] Setting key 19a3eeef172d1533e135388e73cd523a
Aug 11 12:46:55 dist-test-slave-f0tn krb5kdc[14726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452415, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:46:55.381105 15139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:55.364092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:59803 (local address 127.7.111.126:37065)
0811 12:46:55.364266 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:55.364270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:55.365433 (+ 1163us) server_negotiation.cc:407] Connection header received
0811 12:46:55.367223 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:55.367229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:55.367317 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:55.367409 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:55.369456 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.370332 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.372294 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.372723 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.372819 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:55.376037 (+ 3218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:55.376041 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:55.376044 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:55.376076 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:55.378669 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:55.379311 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:55.379314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:55.379317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:55.379436 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:55.379768 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:55.379771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:55.379773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:55.380009 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:55.380123 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:55.380704 (+ 581us) server_negotiation.cc:299] Negotiation successful
0811 12:46:55.380862 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:46:55.382071 15546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:55.364534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:37065 (local address 127.7.111.67:59803)
0811 12:46:55.365260 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:55.365307 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:55.366960 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:55.367658 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:55.367669 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:55.368422 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:55.369213 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:55.369249 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.370503 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.370509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:55.372157 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:55.372178 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.372891 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.372895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:55.373074 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:55.373761 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:46:55.373783 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:55.375843 (+ 2060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:55.378795 (+ 2952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:55.378802 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:55.378813 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:55.379222 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:55.379513 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:46:55.379515 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:46:55.379518 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:46:55.379689 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:46:55.380102 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:55.380107 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:55.380387 (+ 280us) client_negotiation.cc:769] Sending connection context
0811 12:46:55.380623 (+ 236us) client_negotiation.cc:240] Negotiation successful
0811 12:46:55.380985 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":284,"threads_started":1}
I20260811 12:46:55.383424 15542 heartbeater.cc:347] Connected to a master server at 127.7.111.126:37065
I20260811 12:46:55.383759 15542 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:55.384469 15542 heartbeater.cc:511] Master 127.7.111.126:37065 requested a full tablet report, sending...
I20260811 12:46:55.385066 15137 heartbeater.cc:503] Master 127.7.111.126:37065 was elected leader, sending a full tablet report...
I20260811 12:46:55.385963 14800 ts_manager.cc:194] Registered new tserver with Master: 9d1c5485bfb24d15b5bc95dd499ba9a0 (127.7.111.67:37947)
I20260811 12:46:55.386955 14800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:59803
I20260811 12:46:55.396100 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:46:55.412474 15139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:55.400494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52708 (local address 127.7.111.126:37065)
0811 12:46:55.400690 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:55.400697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:55.401214 (+ 517us) server_negotiation.cc:407] Connection header received
0811 12:46:55.401416 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:55.401421 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:55.401486 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:55.401561 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:55.402964 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.403663 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.404809 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.405227 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.405334 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:55.407466 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:55.407471 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:55.407474 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:55.407505 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:55.410164 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:55.410819 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:55.410823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:55.410825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:55.410963 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:55.411261 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:55.411264 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:55.411266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:55.411560 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:55.411802 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:55.412028 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 12:46:55.412187 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 12:46:55.416344 14800 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52708:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:46:55.423209 14800 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:46:55.457566 15565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:55.435252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:37853 (local address 127.0.0.1:46246)
0811 12:46:55.448603 (+ 13351us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:55.448652 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:55.448898 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:55.450064 (+ 1166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:55.450073 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:55.450105 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:55.450574 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:55.450591 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.452513 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.452518 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:55.454727 (+ 2209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:55.454744 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.456527 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.456534 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:55.456725 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:55.456777 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 12:46:55.456914 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 12:46:55.457090 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13060,"mutex_wait_us":22,"thread_start_us":12871,"threads_started":1}
I20260811 12:46:55.458341 15575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:55.435429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:37947 (local address 127.0.0.1:47664)
0811 12:46:55.449358 (+ 13929us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:55.449395 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:55.449606 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:55.450531 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:55.450534 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:55.450558 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:55.451000 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:55.451014 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.452729 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.452736 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:55.455349 (+ 2613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:55.455367 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.457534 (+ 2167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.457542 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:55.457701 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:55.457733 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:46:55.457820 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 12:46:55.457945 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13766,"mutex_wait_us":54,"thread_start_us":13800,"threads_started":1}
I20260811 12:46:55.458487 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:55.437767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41812 (local address 127.7.111.65:40827)
0811 12:46:55.443750 (+ 5983us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:55.443759 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:55.449665 (+ 5906us) server_negotiation.cc:407] Connection header received
0811 12:46:55.450040 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:55.450049 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:55.450266 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:55.450452 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:55.451471 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.452986 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.455612 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.456930 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.457093 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:55.457294 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:55.458079 (+ 785us) server_negotiation.cc:299] Negotiation successful
0811 12:46:55.458217 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5801,"thread_start_us":5638,"threads_started":1}
I20260811 12:46:55.458513 15573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:55.435618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46246 (local address 127.7.111.66:37853)
0811 12:46:55.448098 (+ 12480us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:55.448108 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:55.448866 (+ 758us) server_negotiation.cc:407] Connection header received
0811 12:46:55.449233 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:55.449244 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:55.449551 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:55.449759 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:55.450804 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.452347 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.454881 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.456034 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.456168 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:55.456365 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:55.458002 (+ 1637us) server_negotiation.cc:299] Negotiation successful
0811 12:46:55.458180 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12290,"thread_start_us":12133,"threads_started":1}
I20260811 12:46:55.459097 15570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:55.437065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:40827 (local address 127.0.0.1:41812)
0811 12:46:55.449529 (+ 12464us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:55.449548 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:55.449781 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:55.450779 (+ 998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:55.450784 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:55.450815 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:55.451264 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:55.451278 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.453141 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.453147 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:55.455433 (+ 2286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:55.455446 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.457480 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.457486 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:55.457625 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:55.457660 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 12:46:55.457761 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 12:46:55.457885 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12291,"mutex_wait_us":6,"thread_start_us":11946,"threads_started":1}
I20260811 12:46:55.459097 15574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:55.435787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47664 (local address 127.7.111.67:37947)
0811 12:46:55.448606 (+ 12819us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:55.448615 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:55.449500 (+ 885us) server_negotiation.cc:407] Connection header received
0811 12:46:55.449845 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:55.449853 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:55.450075 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:55.450247 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:55.451250 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.452538 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.455545 (+ 3007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.456968 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.457079 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:55.457303 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:55.458637 (+ 1334us) server_negotiation.cc:299] Negotiation successful
0811 12:46:55.458798 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12634,"thread_start_us":12830,"threads_started":1}
I20260811 12:46:55.462442 15028 tablet_service.cc:1467] Processing CreateTablet for tablet d209c8666d2c44fc8c01c083255ac04e (DEFAULT_TABLE table=test-table [id=4a7354534af946ddb7e3aa6dd028e75b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:55.462610 15444 tablet_service.cc:1467] Processing CreateTablet for tablet d209c8666d2c44fc8c01c083255ac04e (DEFAULT_TABLE table=test-table [id=4a7354534af946ddb7e3aa6dd028e75b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:55.462572 15240 tablet_service.cc:1467] Processing CreateTablet for tablet d209c8666d2c44fc8c01c083255ac04e (DEFAULT_TABLE table=test-table [id=4a7354534af946ddb7e3aa6dd028e75b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:55.463728 15028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d209c8666d2c44fc8c01c083255ac04e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:55.464005 15444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d209c8666d2c44fc8c01c083255ac04e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:55.464059 15240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d209c8666d2c44fc8c01c083255ac04e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:55.479149 15577 tablet_bootstrap.cc:492] T d209c8666d2c44fc8c01c083255ac04e P a78d91862daa43be89d40d792aa9c020: Bootstrap starting.
I20260811 12:46:55.480592 15580 tablet_bootstrap.cc:492] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0: Bootstrap starting.
I20260811 12:46:55.483911 15577 tablet_bootstrap.cc:654] T d209c8666d2c44fc8c01c083255ac04e P a78d91862daa43be89d40d792aa9c020: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:55.483971 15580 tablet_bootstrap.cc:654] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:55.484907 15578 tablet_bootstrap.cc:492] T d209c8666d2c44fc8c01c083255ac04e P 03b0a4d5eb2c46bd90f98d917384d677: Bootstrap starting.
I20260811 12:46:55.485278 15580 log.cc:824] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0: Log is configured to *not* fsync() on all Append() calls
I20260811 12:46:55.486455 15577 log.cc:824] T d209c8666d2c44fc8c01c083255ac04e P a78d91862daa43be89d40d792aa9c020: Log is configured to *not* fsync() on all Append() calls
I20260811 12:46:55.488582 15580 tablet_bootstrap.cc:492] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0: No bootstrap required, opened a new log
I20260811 12:46:55.488631 15578 tablet_bootstrap.cc:654] T d209c8666d2c44fc8c01c083255ac04e P 03b0a4d5eb2c46bd90f98d917384d677: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:55.488885 15580 ts_tablet_manager.cc:1397] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260811 12:46:55.489697 15577 tablet_bootstrap.cc:492] T d209c8666d2c44fc8c01c083255ac04e P a78d91862daa43be89d40d792aa9c020: No bootstrap required, opened a new log
I20260811 12:46:55.489781 15578 log.cc:824] T d209c8666d2c44fc8c01c083255ac04e P 03b0a4d5eb2c46bd90f98d917384d677: Log is configured to *not* fsync() on all Append() calls
I20260811 12:46:55.489960 15577 ts_tablet_manager.cc:1397] T d209c8666d2c44fc8c01c083255ac04e P a78d91862daa43be89d40d792aa9c020: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.006s
I20260811 12:46:55.492377 15578 tablet_bootstrap.cc:492] T d209c8666d2c44fc8c01c083255ac04e P 03b0a4d5eb2c46bd90f98d917384d677: No bootstrap required, opened a new log
I20260811 12:46:55.492646 15578 ts_tablet_manager.cc:1397] T d209c8666d2c44fc8c01c083255ac04e P 03b0a4d5eb2c46bd90f98d917384d677: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260811 12:46:55.499794 15580 raft_consensus.cc:348] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b0a4d5eb2c46bd90f98d917384d677" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37853 } } peers { permanent_uuid: "9d1c5485bfb24d15b5bc95dd499ba9a0" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 37947 } } peers { permanent_uuid: "a78d91862daa43be89d40d792aa9c020" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40827 } }
I20260811 12:46:55.500268 15580 raft_consensus.cc:374] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:55.500384 15580 raft_consensus.cc:729] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d1c5485bfb24d15b5bc95dd499ba9a0, State: Initialized, Role: FOLLOWER
I20260811 12:46:55.500959 15578 raft_consensus.cc:348] T d209c8666d2c44fc8c01c083255ac04e P 03b0a4d5eb2c46bd90f98d917384d677 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b0a4d5eb2c46bd90f98d917384d677" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37853 } } peers { permanent_uuid: "9d1c5485bfb24d15b5bc95dd499ba9a0" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 37947 } } peers { permanent_uuid: "a78d91862daa43be89d40d792aa9c020" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40827 } }
I20260811 12:46:55.501061 15580 consensus_queue.cc:260] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b0a4d5eb2c46bd90f98d917384d677" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37853 } } peers { permanent_uuid: "9d1c5485bfb24d15b5bc95dd499ba9a0" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 37947 } } peers { permanent_uuid: "a78d91862daa43be89d40d792aa9c020" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40827 } }
I20260811 12:46:55.501307 15578 raft_consensus.cc:374] T d209c8666d2c44fc8c01c083255ac04e P 03b0a4d5eb2c46bd90f98d917384d677 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:55.501188 15577 raft_consensus.cc:348] T d209c8666d2c44fc8c01c083255ac04e P a78d91862daa43be89d40d792aa9c020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b0a4d5eb2c46bd90f98d917384d677" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37853 } } peers { permanent_uuid: "9d1c5485bfb24d15b5bc95dd499ba9a0" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 37947 } } peers { permanent_uuid: "a78d91862daa43be89d40d792aa9c020" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40827 } }
I20260811 12:46:55.501461 15578 raft_consensus.cc:729] T d209c8666d2c44fc8c01c083255ac04e P 03b0a4d5eb2c46bd90f98d917384d677 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03b0a4d5eb2c46bd90f98d917384d677, State: Initialized, Role: FOLLOWER
I20260811 12:46:55.501638 15577 raft_consensus.cc:374] T d209c8666d2c44fc8c01c083255ac04e P a78d91862daa43be89d40d792aa9c020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:55.501847 15577 raft_consensus.cc:729] T d209c8666d2c44fc8c01c083255ac04e P a78d91862daa43be89d40d792aa9c020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a78d91862daa43be89d40d792aa9c020, State: Initialized, Role: FOLLOWER
I20260811 12:46:55.502205 15578 consensus_queue.cc:260] T d209c8666d2c44fc8c01c083255ac04e P 03b0a4d5eb2c46bd90f98d917384d677 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b0a4d5eb2c46bd90f98d917384d677" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37853 } } peers { permanent_uuid: "9d1c5485bfb24d15b5bc95dd499ba9a0" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 37947 } } peers { permanent_uuid: "a78d91862daa43be89d40d792aa9c020" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40827 } }
I20260811 12:46:55.502473 15577 consensus_queue.cc:260] T d209c8666d2c44fc8c01c083255ac04e P a78d91862daa43be89d40d792aa9c020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b0a4d5eb2c46bd90f98d917384d677" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37853 } } peers { permanent_uuid: "9d1c5485bfb24d15b5bc95dd499ba9a0" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 37947 } } peers { permanent_uuid: "a78d91862daa43be89d40d792aa9c020" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40827 } }
I20260811 12:46:55.503389 15337 heartbeater.cc:503] Master 127.7.111.126:37065 was elected leader, sending a full tablet report...
I20260811 12:46:55.504186 15578 ts_tablet_manager.cc:1428] T d209c8666d2c44fc8c01c083255ac04e P 03b0a4d5eb2c46bd90f98d917384d677: Time spent starting tablet: real 0.011s user 0.007s sys 0.005s
I20260811 12:46:55.505155 15542 heartbeater.cc:503] Master 127.7.111.126:37065 was elected leader, sending a full tablet report...
I20260811 12:46:55.505872 15580 ts_tablet_manager.cc:1428] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0: Time spent starting tablet: real 0.017s user 0.018s sys 0.001s
I20260811 12:46:55.508538 15577 ts_tablet_manager.cc:1428] T d209c8666d2c44fc8c01c083255ac04e P a78d91862daa43be89d40d792aa9c020: Time spent starting tablet: real 0.018s user 0.017s sys 0.000s
W20260811 12:46:55.604175 15338 tablet.cc:2369] T d209c8666d2c44fc8c01c083255ac04e P 03b0a4d5eb2c46bd90f98d917384d677: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:46:55.606824 15138 tablet.cc:2369] T d209c8666d2c44fc8c01c083255ac04e P a78d91862daa43be89d40d792aa9c020: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:46:55.613085 15545 tablet.cc:2369] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:46:55.625764 15588 raft_consensus.cc:482] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:46:55.626047 15588 raft_consensus.cc:504] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b0a4d5eb2c46bd90f98d917384d677" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37853 } } peers { permanent_uuid: "9d1c5485bfb24d15b5bc95dd499ba9a0" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 37947 } } peers { permanent_uuid: "a78d91862daa43be89d40d792aa9c020" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40827 } }
I20260811 12:46:55.627777 15588 leader_election.cc:288] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03b0a4d5eb2c46bd90f98d917384d677 (127.7.111.66:37853), a78d91862daa43be89d40d792aa9c020 (127.7.111.65:40827)
I20260811 12:46:55.633502 15573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:55.628323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:44721 (local address 127.7.111.66:37853)
0811 12:46:55.628529 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:55.628534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:55.628556 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:46:55.628753 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:55.628757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:55.628817 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:55.628890 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:55.629319 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.630122 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.631836 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.632616 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.632759 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:55.632835 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:55.633097 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 12:46:55.633232 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 12:46:55.633505 15546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:55.628258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:37853 (local address 127.7.111.67:44721)
0811 12:46:55.628449 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:55.628469 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:55.628609 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:55.628929 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:55.628933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:55.628965 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:55.629179 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:55.629189 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.630265 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.630271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:55.631666 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:55.631682 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.632766 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.632771 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:55.632954 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:55.632979 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 12:46:55.633060 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 12:46:55.633192 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 12:46:55.634282 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:55.628376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:32991 (local address 127.7.111.65:40827)
0811 12:46:55.628581 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:55.628587 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:55.629338 (+ 751us) server_negotiation.cc:407] Connection header received
0811 12:46:55.629519 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:55.629524 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:55.629595 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:55.629682 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:46:55.630198 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.630970 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.632628 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.633400 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.633560 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:55.633633 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:55.633963 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 12:46:55.634071 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:46:55.634362 15596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:55.628265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:40827 (local address 127.7.111.67:32991)
0811 12:46:55.629229 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:55.629243 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:55.629364 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:55.629699 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:55.629703 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:55.629726 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:46:55.630033 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:55.630042 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.631120 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.631124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:55.632472 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:55.632484 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.633598 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.633604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:55.633786 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:55.633821 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 12:46:55.633918 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 12:46:55.634043 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"mutex_wait_us":221,"thread_start_us":414,"threads_started":1}
I20260811 12:46:55.634750 15271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d209c8666d2c44fc8c01c083255ac04e" candidate_uuid: "9d1c5485bfb24d15b5bc95dd499ba9a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03b0a4d5eb2c46bd90f98d917384d677" is_pre_election: true
I20260811 12:46:55.635319 15271 raft_consensus.cc:2454] T d209c8666d2c44fc8c01c083255ac04e P 03b0a4d5eb2c46bd90f98d917384d677 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d1c5485bfb24d15b5bc95dd499ba9a0 in term 0.
I20260811 12:46:55.635485 15057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d209c8666d2c44fc8c01c083255ac04e" candidate_uuid: "9d1c5485bfb24d15b5bc95dd499ba9a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a78d91862daa43be89d40d792aa9c020" is_pre_election: true
I20260811 12:46:55.636019 15057 raft_consensus.cc:2454] T d209c8666d2c44fc8c01c083255ac04e P a78d91862daa43be89d40d792aa9c020 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d1c5485bfb24d15b5bc95dd499ba9a0 in term 0.
I20260811 12:46:55.636169 15373 leader_election.cc:302] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03b0a4d5eb2c46bd90f98d917384d677, 9d1c5485bfb24d15b5bc95dd499ba9a0; no voters:
I20260811 12:46:55.636574 15588 raft_consensus.cc:2781] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:46:55.636731 15588 raft_consensus.cc:482] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:46:55.636852 15588 raft_consensus.cc:3037] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:55.639019 15588 raft_consensus.cc:504] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b0a4d5eb2c46bd90f98d917384d677" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37853 } } peers { permanent_uuid: "9d1c5485bfb24d15b5bc95dd499ba9a0" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 37947 } } peers { permanent_uuid: "a78d91862daa43be89d40d792aa9c020" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40827 } }
I20260811 12:46:55.639596 15588 leader_election.cc:288] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0 [CANDIDATE]: Term 1 election: Requested vote from peers 03b0a4d5eb2c46bd90f98d917384d677 (127.7.111.66:37853), a78d91862daa43be89d40d792aa9c020 (127.7.111.65:40827)
I20260811 12:46:55.639950 15271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d209c8666d2c44fc8c01c083255ac04e" candidate_uuid: "9d1c5485bfb24d15b5bc95dd499ba9a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03b0a4d5eb2c46bd90f98d917384d677"
I20260811 12:46:55.640125 15271 raft_consensus.cc:3037] T d209c8666d2c44fc8c01c083255ac04e P 03b0a4d5eb2c46bd90f98d917384d677 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:55.640103 15057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d209c8666d2c44fc8c01c083255ac04e" candidate_uuid: "9d1c5485bfb24d15b5bc95dd499ba9a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a78d91862daa43be89d40d792aa9c020"
I20260811 12:46:55.640291 15057 raft_consensus.cc:3037] T d209c8666d2c44fc8c01c083255ac04e P a78d91862daa43be89d40d792aa9c020 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:55.642330 15271 raft_consensus.cc:2454] T d209c8666d2c44fc8c01c083255ac04e P 03b0a4d5eb2c46bd90f98d917384d677 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d1c5485bfb24d15b5bc95dd499ba9a0 in term 1.
I20260811 12:46:55.642524 15057 raft_consensus.cc:2454] T d209c8666d2c44fc8c01c083255ac04e P a78d91862daa43be89d40d792aa9c020 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d1c5485bfb24d15b5bc95dd499ba9a0 in term 1.
I20260811 12:46:55.642685 15373 leader_election.cc:302] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03b0a4d5eb2c46bd90f98d917384d677, 9d1c5485bfb24d15b5bc95dd499ba9a0; no voters:
I20260811 12:46:55.642980 15588 raft_consensus.cc:2781] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:46:55.643416 15588 raft_consensus.cc:686] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0 [term 1 LEADER]: Becoming Leader. State: Replica: 9d1c5485bfb24d15b5bc95dd499ba9a0, State: Running, Role: LEADER
I20260811 12:46:55.644109 15588 consensus_queue.cc:237] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b0a4d5eb2c46bd90f98d917384d677" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37853 } } peers { permanent_uuid: "9d1c5485bfb24d15b5bc95dd499ba9a0" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 37947 } } peers { permanent_uuid: "a78d91862daa43be89d40d792aa9c020" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40827 } }
I20260811 12:46:55.648830 14800 catalog_manager.cc:5495] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d1c5485bfb24d15b5bc95dd499ba9a0 (127.7.111.67). New cstate: current_term: 1 leader_uuid: "9d1c5485bfb24d15b5bc95dd499ba9a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b0a4d5eb2c46bd90f98d917384d677" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d1c5485bfb24d15b5bc95dd499ba9a0" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 37947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a78d91862daa43be89d40d792aa9c020" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40827 } health_report { overall_health: UNKNOWN } } }
I20260811 12:46:55.704173 15574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:55.699172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47676 (local address 127.7.111.67:37947)
0811 12:46:55.699373 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:55.699377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:55.699395 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:46:55.699562 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:55.699565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:55.699648 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:55.699782 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:46:55.700061 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.700798 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.702251 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.702654 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.702759 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:55.703159 (+ 400us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:46:55.703667 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:46:55.703810 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:55.703848 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 12:46:55.703916 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:46:55.712845 15057 raft_consensus.cc:1261] T d209c8666d2c44fc8c01c083255ac04e P a78d91862daa43be89d40d792aa9c020 [term 1 FOLLOWER]: Refusing update from remote peer 9d1c5485bfb24d15b5bc95dd499ba9a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:46:55.712844 15271 raft_consensus.cc:1261] T d209c8666d2c44fc8c01c083255ac04e P 03b0a4d5eb2c46bd90f98d917384d677 [term 1 FOLLOWER]: Refusing update from remote peer 9d1c5485bfb24d15b5bc95dd499ba9a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:46:55.713986 15588 consensus_queue.cc:1035] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a78d91862daa43be89d40d792aa9c020" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:46:55.714301 15597 consensus_queue.cc:1035] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03b0a4d5eb2c46bd90f98d917384d677" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:46:55.729880 15602 mvcc.cc:205] Tried to move back new op lower bound from 7317309094748168192 to 7317309094481956864. Current Snapshot: MvccSnapshot[applied={T|T < 7317309094748168192}]
I20260811 12:46:55.732203 15604 mvcc.cc:205] Tried to move back new op lower bound from 7317309094748168192 to 7317309094481956864. Current Snapshot: MvccSnapshot[applied={T|T < 7317309094748168192}]
I20260811 12:46:55.744232 14800 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:46:55.744530 14800 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:46:55.748332 14800 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 39b1e42232754517be52045a3df5e428: Sending DeleteTablet for 3 replicas of tablet d209c8666d2c44fc8c01c083255ac04e
I20260811 12:46:55.749610 15240 tablet_service.cc:1514] Processing DeleteTablet for tablet d209c8666d2c44fc8c01c083255ac04e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:55 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:46246
I20260811 12:46:55.749692 15444 tablet_service.cc:1514] Processing DeleteTablet for tablet d209c8666d2c44fc8c01c083255ac04e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:55 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:47664
I20260811 12:46:55.750936 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 14919
I20260811 12:46:55.750985 15028 tablet_service.cc:1514] Processing DeleteTablet for tablet d209c8666d2c44fc8c01c083255ac04e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:55 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:41812
I20260811 12:46:55.762418 15619 tablet_replica.cc:330] T d209c8666d2c44fc8c01c083255ac04e P a78d91862daa43be89d40d792aa9c020: stopping tablet replica
I20260811 12:46:55.762926 15619 raft_consensus.cc:2229] T d209c8666d2c44fc8c01c083255ac04e P a78d91862daa43be89d40d792aa9c020 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:46:55.763495 15619 raft_consensus.cc:2258] T d209c8666d2c44fc8c01c083255ac04e P a78d91862daa43be89d40d792aa9c020 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:46:55.784547 15616 tablet_replica.cc:330] T d209c8666d2c44fc8c01c083255ac04e P 03b0a4d5eb2c46bd90f98d917384d677: stopping tablet replica
I20260811 12:46:55.786634 15619 ts_tablet_manager.cc:1905] T d209c8666d2c44fc8c01c083255ac04e P a78d91862daa43be89d40d792aa9c020: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:46:55.787240 15616 raft_consensus.cc:2229] T d209c8666d2c44fc8c01c083255ac04e P 03b0a4d5eb2c46bd90f98d917384d677 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:46:55.788859 15616 raft_consensus.cc:2258] T d209c8666d2c44fc8c01c083255ac04e P 03b0a4d5eb2c46bd90f98d917384d677 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:46:55.789948 15618 tablet_replica.cc:330] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0: stopping tablet replica
I20260811 12:46:55.790436 15618 raft_consensus.cc:2229] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:46:55.791265 15618 raft_consensus.cc:2258] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:46:55.793502 15618 ts_tablet_manager.cc:1905] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:46:55.794691 15616 ts_tablet_manager.cc:1905] T d209c8666d2c44fc8c01c083255ac04e P 03b0a4d5eb2c46bd90f98d917384d677: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:46:55.796201 15619 ts_tablet_manager.cc:1918] T d209c8666d2c44fc8c01c083255ac04e P a78d91862daa43be89d40d792aa9c020: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:46:55.796451 15619 log.cc:1196] T d209c8666d2c44fc8c01c083255ac04e P a78d91862daa43be89d40d792aa9c020: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/d209c8666d2c44fc8c01c083255ac04e
I20260811 12:46:55.797086 15619 ts_tablet_manager.cc:1939] T d209c8666d2c44fc8c01c083255ac04e P a78d91862daa43be89d40d792aa9c020: Deleting consensus metadata
I20260811 12:46:55.799310 14778 catalog_manager.cc:4841] TS a78d91862daa43be89d40d792aa9c020 (127.7.111.65:40827): tablet d209c8666d2c44fc8c01c083255ac04e (table test-table [id=4a7354534af946ddb7e3aa6dd028e75b]) successfully deleted
Aug 11 12:46:55 dist-test-slave-f0tn krb5kdc[14726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.65@KRBTEST.COM
I20260811 12:46:55.805392 15616 ts_tablet_manager.cc:1918] T d209c8666d2c44fc8c01c083255ac04e P 03b0a4d5eb2c46bd90f98d917384d677: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:46:55.805728 15616 log.cc:1196] T d209c8666d2c44fc8c01c083255ac04e P 03b0a4d5eb2c46bd90f98d917384d677: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/d209c8666d2c44fc8c01c083255ac04e
I20260811 12:46:55.806344 15616 ts_tablet_manager.cc:1939] T d209c8666d2c44fc8c01c083255ac04e P 03b0a4d5eb2c46bd90f98d917384d677: Deleting consensus metadata
I20260811 12:46:55.807910 15618 ts_tablet_manager.cc:1918] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:46:55.808146 15618 log.cc:1196] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/d209c8666d2c44fc8c01c083255ac04e
I20260811 12:46:55.808444 14780 catalog_manager.cc:4841] TS 03b0a4d5eb2c46bd90f98d917384d677 (127.7.111.66:37853): tablet d209c8666d2c44fc8c01c083255ac04e (table test-table [id=4a7354534af946ddb7e3aa6dd028e75b]) successfully deleted
I20260811 12:46:55.808708 15618 ts_tablet_manager.cc:1939] T d209c8666d2c44fc8c01c083255ac04e P 9d1c5485bfb24d15b5bc95dd499ba9a0: Deleting consensus metadata
I20260811 12:46:55.809949 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:55.752256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41816 (local address 127.7.111.65:40827)
0811 12:46:55.752444 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:55.752451 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:55.776029 (+ 23578us) server_negotiation.cc:407] Connection header received
0811 12:46:55.781854 (+ 5825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:55.781861 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:55.781965 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:55.782048 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:55.784836 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.785953 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.789367 (+ 3414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.789976 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.790074 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:55.802146 (+ 12072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:55.802164 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:55.802186 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:55.802243 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:55.806175 (+ 3932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:55.807120 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:55.807131 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:55.807135 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:55.807327 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:55.807834 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:55.807839 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:55.807842 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:55.808355 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:55.808669 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:55.809304 (+ 635us) server_negotiation.cc:299] Negotiation successful
0811 12:46:55.809576 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:46:55.810487 14777 catalog_manager.cc:4841] TS 9d1c5485bfb24d15b5bc95dd499ba9a0 (127.7.111.67:37947): tablet d209c8666d2c44fc8c01c083255ac04e (table test-table [id=4a7354534af946ddb7e3aa6dd028e75b]) successfully deleted
I20260811 12:46:55.810653 15131 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:55.901455 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 14919
I20260811 12:46:55.943861 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 15143
Aug 11 12:46:55 dist-test-slave-f0tn krb5kdc[14726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.66@KRBTEST.COM
I20260811 12:46:55.959230 15573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:55.944736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46260 (local address 127.7.111.66:37853)
0811 12:46:55.944943 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:55.944949 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:55.944969 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:46:55.945097 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:55.945101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:55.945169 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:55.945264 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:55.946595 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.947347 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.948564 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:55.948957 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:55.949069 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:55.953442 (+ 4373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:55.953456 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:55.953473 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:55.953509 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:55.956254 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:55.957006 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:55.957012 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:55.957015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:55.957138 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:55.957631 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:55.957634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:55.957636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:55.958012 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:55.958254 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:55.958564 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 12:46:55.958725 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 12:46:55.960119 15331 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:56.114740 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 15143
I20260811 12:46:56.144373 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 15345
Aug 11 12:46:56 dist-test-slave-f0tn krb5kdc[14726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.67@KRBTEST.COM
I20260811 12:46:56.163342 15574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:56.145205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47678 (local address 127.7.111.67:37947)
0811 12:46:56.145425 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:56.145432 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:56.145451 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:46:56.145649 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:56.145653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:56.145722 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:56.145802 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:56.147607 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:56.148453 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:56.150366 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:56.150883 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:56.150993 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:56.156704 (+ 5711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:56.156722 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:56.156741 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:46:56.156783 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:56.160050 (+ 3267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:56.160935 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:56.160944 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:56.160946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:56.161113 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:46:56.161770 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:46:56.161774 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:46:56.161776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:46:56.162184 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:56.162428 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:56.162821 (+ 393us) server_negotiation.cc:299] Negotiation successful
0811 12:46:56.163008 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:46:56.164078 15536 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:56.289254 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 15345
I20260811 12:46:56.316367 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 14751
I20260811 12:46:56.317548 14854 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:56.427642 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 14751
2026-08-11T12:46:56Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4542 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T12:46:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:46:56Z Disabled control of system clock
I20260811 12:46:56.509009 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:46155
--webserver_interface=127.7.111.126
--webserver_port=0
--builtin_ntp_servers=127.7.111.84:33291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:46155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 12:46:56.837716 15631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:56.838034 15631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:56.838142 15631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:56.845670 15631 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:46:56.845774 15631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:56.845817 15631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:56.845875 15631 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:46:56.845923 15631 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:46:56.855227 15631 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:33291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:46155
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:46155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:56.856472 15631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:56.858258 15631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:56.870149 15644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:56.870023 15651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:56.870222 15645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:56.871418 15631 server_base.cc:1034] running on GCE node
I20260811 12:46:56.872174 15631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:56.873581 15631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:56.874792 15631 hybrid_clock.cc:648] HybridClock initialized: now 1786452416874735 us; error 47 us; skew 500 ppm
I20260811 12:46:56.875133 15631 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:46:56.881310 15631 webserver.cc:460] Webserver started at http://127.7.111.126:42455/ using document root <none> and password file <none>
I20260811 12:46:56.882074 15631 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:56.882169 15631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:56.882490 15631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:56.885515 15631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "fd5df18c20da4303bac88b841cd6c5db"
format_stamp: "Formatted at 2026-08-11 12:46:56 on dist-test-slave-f0tn"
server_key: "1eb893c3f9b0a3b2a1b978a15a2817c0"
server_key_iv: "5f9b69ec6631228f9fb44225ba2cedc9"
server_key_version: "encryptionkey@0"
I20260811 12:46:56.886324 15631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "fd5df18c20da4303bac88b841cd6c5db"
format_stamp: "Formatted at 2026-08-11 12:46:56 on dist-test-slave-f0tn"
server_key: "1eb893c3f9b0a3b2a1b978a15a2817c0"
server_key_iv: "5f9b69ec6631228f9fb44225ba2cedc9"
server_key_version: "encryptionkey@0"
I20260811 12:46:56.891418 15631 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:46:56.894745 15663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:56.896042 15631 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:46:56.896220 15631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "fd5df18c20da4303bac88b841cd6c5db"
format_stamp: "Formatted at 2026-08-11 12:46:56 on dist-test-slave-f0tn"
server_key: "1eb893c3f9b0a3b2a1b978a15a2817c0"
server_key_iv: "5f9b69ec6631228f9fb44225ba2cedc9"
server_key_version: "encryptionkey@0"
I20260811 12:46:56.896418 15631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:56.902221 15631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:56.902983 15631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:56.903260 15631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:56.920464 15631 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:46155
I20260811 12:46:56.920485 15747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:46155 every 8 connection(s)
I20260811 12:46:56.921891 15631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:46:56.925691 15749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:56.930783 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 15631
I20260811 12:46:56.930944 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:46:56.931586 7613 external_mini_cluster.cc:1428] Setting key 3492b9e9d39a89988b93528b70023dea
I20260811 12:46:56.936380 15749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db: Bootstrap starting.
I20260811 12:46:56.939772 15756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:56.933593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44390 (local address 127.7.111.126:46155)
0811 12:46:56.934633 (+ 1040us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:56.934651 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:56.934695 (+ 44us) server_negotiation.cc:407] Connection header received
0811 12:46:56.936301 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:56.936323 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:56.936755 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:56.937071 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:56.938236 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:56.938242 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:56.938254 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:46:56.938269 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:56.938327 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:46:56.938342 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:56.938514 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:56.938647 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 12:46:56.938899 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":364,"threads_started":1}
I20260811 12:46:56.939957 15749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:56.941296 15749 log.cc:824] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db: Log is configured to *not* fsync() on all Append() calls
I20260811 12:46:56.944013 15749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db: No bootstrap required, opened a new log
I20260811 12:46:56.951283 15749 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5df18c20da4303bac88b841cd6c5db" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46155 } }
I20260811 12:46:56.951608 15749 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:56.951704 15749 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd5df18c20da4303bac88b841cd6c5db, State: Initialized, Role: FOLLOWER
I20260811 12:46:56.952226 15749 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5df18c20da4303bac88b841cd6c5db" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46155 } }
I20260811 12:46:56.952435 15749 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:46:56.952574 15749 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:46:56.952731 15749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:56.954909 15749 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5df18c20da4303bac88b841cd6c5db" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46155 } }
I20260811 12:46:56.955371 15749 leader_election.cc:302] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd5df18c20da4303bac88b841cd6c5db; no voters:
I20260811 12:46:56.955890 15749 leader_election.cc:288] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:46:56.956020 15758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:46:56.956727 15758 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db [term 1 LEADER]: Becoming Leader. State: Replica: fd5df18c20da4303bac88b841cd6c5db, State: Running, Role: LEADER
I20260811 12:46:56.957234 15758 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5df18c20da4303bac88b841cd6c5db" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46155 } }
I20260811 12:46:56.957882 15749 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:46:56.959746 15763 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd5df18c20da4303bac88b841cd6c5db. Latest consensus state: current_term: 1 leader_uuid: "fd5df18c20da4303bac88b841cd6c5db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5df18c20da4303bac88b841cd6c5db" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46155 } } }
I20260811 12:46:56.960150 15763 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db [sys.catalog]: This master's current role is: LEADER
I20260811 12:46:56.961261 15759 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd5df18c20da4303bac88b841cd6c5db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5df18c20da4303bac88b841cd6c5db" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46155 } } }
I20260811 12:46:56.962103 15759 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db [sys.catalog]: This master's current role is: LEADER
I20260811 12:46:56.962190 15773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:46:56.966859 15773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:46:56.975250 15773 catalog_manager.cc:1347] Generated new cluster ID: b1ee3dc9e3c24899ae7b68f4170ccf8b
I20260811 12:46:56.975371 15773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:46:56.993983 15773 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:46:56.994434 15773 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:46:57.001724 15773 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db: Generated new TSK 0
I20260811 12:46:57.002593 15773 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:46:57.015508 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:46155
--builtin_ntp_servers=127.7.111.84:33291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 12:46:57.363103 15786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:57.363421 15786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:57.363574 15786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:57.371207 15786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:57.371331 15786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:57.371502 15786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:46:57.381134 15786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:33291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:46155
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:57.382661 15786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:57.384699 15786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:57.398298 15796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:57.398265 15794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:57.398524 15800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:57.399219 15786 server_base.cc:1034] running on GCE node
I20260811 12:46:57.399962 15786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:57.401012 15786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:57.402223 15786 hybrid_clock.cc:648] HybridClock initialized: now 1786452417402148 us; error 57 us; skew 500 ppm
I20260811 12:46:57.402565 15786 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:46:57.404851 15786 webserver.cc:460] Webserver started at http://127.7.111.65:33141/ using document root <none> and password file <none>
I20260811 12:46:57.405609 15786 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:57.405717 15786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:57.406028 15786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:57.409004 15786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "7101e1c87f474d989066320831d49e1e"
format_stamp: "Formatted at 2026-08-11 12:46:57 on dist-test-slave-f0tn"
server_key: "751cb2329202e1a04662e00abe2bb89f"
server_key_iv: "87901b349bdd650dccb7a53a0ed33725"
server_key_version: "encryptionkey@0"
I20260811 12:46:57.409827 15786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "7101e1c87f474d989066320831d49e1e"
format_stamp: "Formatted at 2026-08-11 12:46:57 on dist-test-slave-f0tn"
server_key: "751cb2329202e1a04662e00abe2bb89f"
server_key_iv: "87901b349bdd650dccb7a53a0ed33725"
server_key_version: "encryptionkey@0"
I20260811 12:46:57.414824 15786 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 12:46:57.418277 15807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:57.419566 15786 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:46:57.419710 15786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "7101e1c87f474d989066320831d49e1e"
format_stamp: "Formatted at 2026-08-11 12:46:57 on dist-test-slave-f0tn"
server_key: "751cb2329202e1a04662e00abe2bb89f"
server_key_iv: "87901b349bdd650dccb7a53a0ed33725"
server_key_version: "encryptionkey@0"
I20260811 12:46:57.419925 15786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:57.425441 15786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:57.426218 15786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:57.426477 15786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:57.427380 15786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:46:57.428766 15786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:57.428861 15786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:57.428972 15786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:57.429054 15786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:57.459933 15786 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:40063
I20260811 12:46:57.459990 15976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:40063 every 8 connection(s)
I20260811 12:46:57.461433 15786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:46:57.469902 15979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:57.464273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:35779 (local address 127.7.111.126:46155)
0811 12:46:57.464754 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:57.464758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:57.465955 (+ 1197us) server_negotiation.cc:407] Connection header received
0811 12:46:57.467675 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:57.467682 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:57.467751 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:57.467848 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:57.469113 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:57.469119 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:57.469124 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:46:57.469143 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:57.469187 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:46:57.469207 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:57.469319 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:57.469541 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 12:46:57.469610 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":250,"threads_started":1}
I20260811 12:46:57.470836 15980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:57.464809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:46155 (local address 127.7.111.65:35779)
0811 12:46:57.465794 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:57.465837 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:57.467395 (+ 1558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:57.468104 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:57.468112 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:57.468211 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:57.468821 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:46:57.468823 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:57.468861 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:46:57.468945 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:57.469297 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:57.469308 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:57.469316 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:46:57.469606 (+ 290us) client_negotiation.cc:240] Negotiation successful
0811 12:46:57.469880 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":291,"threads_started":1}
I20260811 12:46:57.471220 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 15786
I20260811 12:46:57.471400 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:46:57.472184 7613 external_mini_cluster.cc:1428] Setting key 5f369818b828cb8a6c48ca20940192b5
I20260811 12:46:57.472352 15977 heartbeater.cc:347] Connected to a master server at 127.7.111.126:46155
I20260811 12:46:57.472627 15977 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:57.473212 15977 heartbeater.cc:511] Master 127.7.111.126:46155 requested a full tablet report, sending...
I20260811 12:46:57.475176 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:46155
--builtin_ntp_servers=127.7.111.84:33291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 12:46:57.475549 15689 ts_manager.cc:194] Registered new tserver with Master: 7101e1c87f474d989066320831d49e1e (127.7.111.65:40063)
W20260811 12:46:57.807543 15981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:57.807811 15981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:57.807914 15981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:57.814994 15981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:57.815104 15981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:57.815220 15981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:46:57.824239 15981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:33291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:46155
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:57.825466 15981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:57.827240 15981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:57.839792 15995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:57.839841 15992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:57.840018 15991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:57.840723 15981 server_base.cc:1034] running on GCE node
I20260811 12:46:57.841320 15981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:57.842253 15981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:57.843441 15981 hybrid_clock.cc:648] HybridClock initialized: now 1786452417843391 us; error 43 us; skew 500 ppm
I20260811 12:46:57.843730 15981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:46:57.845822 15981 webserver.cc:460] Webserver started at http://127.7.111.66:39413/ using document root <none> and password file <none>
I20260811 12:46:57.846460 15981 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:57.846554 15981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:57.846865 15981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:57.849617 15981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "7f123c21682f466c997ddd184969f5f8"
format_stamp: "Formatted at 2026-08-11 12:46:57 on dist-test-slave-f0tn"
server_key: "35cd69675899b3c6cf64a7f7799cb8c0"
server_key_iv: "326f5681357e52149641c2f07057723a"
server_key_version: "encryptionkey@0"
I20260811 12:46:57.850343 15981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "7f123c21682f466c997ddd184969f5f8"
format_stamp: "Formatted at 2026-08-11 12:46:57 on dist-test-slave-f0tn"
server_key: "35cd69675899b3c6cf64a7f7799cb8c0"
server_key_iv: "326f5681357e52149641c2f07057723a"
server_key_version: "encryptionkey@0"
I20260811 12:46:57.855103 15981 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 12:46:57.858310 16010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:57.859549 15981 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:46:57.859681 15981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "7f123c21682f466c997ddd184969f5f8"
format_stamp: "Formatted at 2026-08-11 12:46:57 on dist-test-slave-f0tn"
server_key: "35cd69675899b3c6cf64a7f7799cb8c0"
server_key_iv: "326f5681357e52149641c2f07057723a"
server_key_version: "encryptionkey@0"
I20260811 12:46:57.859879 15981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:57.865163 15981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:57.865906 15981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:57.866194 15981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:57.867097 15981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:46:57.868386 15981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:57.868467 15981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:57.868552 15981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:57.868605 15981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:57.898766 15981 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:44183
I20260811 12:46:57.898857 16184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:44183 every 8 connection(s)
I20260811 12:46:57.900094 15981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:46:57.908841 15979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:57.902797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:43991 (local address 127.7.111.126:46155)
0811 12:46:57.902983 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:57.902988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:57.904526 (+ 1538us) server_negotiation.cc:407] Connection header received
0811 12:46:57.906445 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:57.906453 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:57.906525 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:57.906632 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:57.908007 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:57.908013 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:57.908016 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:46:57.908036 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:57.908073 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:46:57.908087 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:57.908228 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:57.908420 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 12:46:57.908496 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 12:46:57.908856 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 15981
I20260811 12:46:57.909005 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:46:57.909489 7613 external_mini_cluster.cc:1428] Setting key 1fe7434d72b399ece54e8ddd53b692ea
I20260811 12:46:57.909933 16190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:57.903314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:46155 (local address 127.7.111.66:43991)
0811 12:46:57.904338 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:57.904384 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:57.906141 (+ 1757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:57.906882 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:57.906891 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:57.906986 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:57.907664 (+ 678us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:46:57.907667 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:57.907707 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:46:57.907801 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:57.908189 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:57.908202 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:57.908211 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 12:46:57.908530 (+ 319us) client_negotiation.cc:240] Negotiation successful
0811 12:46:57.908798 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":319,"threads_started":1}
I20260811 12:46:57.911389 16187 heartbeater.cc:347] Connected to a master server at 127.7.111.126:46155
I20260811 12:46:57.911706 16187 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:57.912062 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:46155
--builtin_ntp_servers=127.7.111.84:33291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 12:46:57.912382 16187 heartbeater.cc:511] Master 127.7.111.126:46155 requested a full tablet report, sending...
I20260811 12:46:57.913923 15689 ts_manager.cc:194] Registered new tserver with Master: 7f123c21682f466c997ddd184969f5f8 (127.7.111.66:44183)
W20260811 12:46:58.252767 16194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:58.253053 16194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:58.253171 16194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:58.260573 16194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:58.260675 16194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:58.260820 16194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:46:58.270205 16194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:33291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:46155
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:58.271557 16194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:58.273411 16194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:58.286497 16201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:58.286542 16204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:58.286463 16202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:58.287566 16194 server_base.cc:1034] running on GCE node
I20260811 12:46:58.288194 16194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:58.289146 16194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:58.290352 16194 hybrid_clock.cc:648] HybridClock initialized: now 1786452418290283 us; error 59 us; skew 500 ppm
I20260811 12:46:58.290674 16194 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:46:58.292939 16194 webserver.cc:460] Webserver started at http://127.7.111.67:43209/ using document root <none> and password file <none>
I20260811 12:46:58.293717 16194 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:58.293835 16194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:58.294139 16194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:58.297151 16194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "1a1fe9b4b9d4467c8d13acb79a3ec8f9"
format_stamp: "Formatted at 2026-08-11 12:46:58 on dist-test-slave-f0tn"
server_key: "e2b65ef8f8484eb61a588ac84382bad5"
server_key_iv: "a96785a85af7cf2f0e60f6f0188bb51e"
server_key_version: "encryptionkey@0"
I20260811 12:46:58.298064 16194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "1a1fe9b4b9d4467c8d13acb79a3ec8f9"
format_stamp: "Formatted at 2026-08-11 12:46:58 on dist-test-slave-f0tn"
server_key: "e2b65ef8f8484eb61a588ac84382bad5"
server_key_iv: "a96785a85af7cf2f0e60f6f0188bb51e"
server_key_version: "encryptionkey@0"
I20260811 12:46:58.303946 16194 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:46:58.307668 16213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:58.309055 16194 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:46:58.309229 16194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "1a1fe9b4b9d4467c8d13acb79a3ec8f9"
format_stamp: "Formatted at 2026-08-11 12:46:58 on dist-test-slave-f0tn"
server_key: "e2b65ef8f8484eb61a588ac84382bad5"
server_key_iv: "a96785a85af7cf2f0e60f6f0188bb51e"
server_key_version: "encryptionkey@0"
I20260811 12:46:58.309502 16194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:58.315274 16194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:58.316076 16194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:58.316321 16194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:58.317269 16194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:46:58.318702 16194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:46:58.318806 16194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:58.318888 16194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:46:58.318941 16194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:58.349097 16194 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:36221
I20260811 12:46:58.349161 16403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:36221 every 8 connection(s)
I20260811 12:46:58.350513 16194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:46:58.358451 15979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.353105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:57135 (local address 127.7.111.126:46155)
0811 12:46:58.353315 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:58.353321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:58.354861 (+ 1540us) server_negotiation.cc:407] Connection header received
0811 12:46:58.356463 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:58.356467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:58.356520 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:58.356606 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:58.357855 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:58.357860 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:58.357863 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:46:58.357878 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:58.357911 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:46:58.357923 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:58.358016 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:58.358177 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 12:46:58.358233 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:46:58.359355 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.353772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:46155 (local address 127.7.111.67:57135)
0811 12:46:58.354688 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:58.354742 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:58.356245 (+ 1503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:58.356805 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:58.356812 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:58.356900 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:58.357553 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:46:58.357555 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:58.357594 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:46:58.357681 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:58.358001 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:58.358006 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:58.358011 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 12:46:58.358220 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 12:46:58.358438 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":277,"threads_started":1}
I20260811 12:46:58.360365 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 16194
I20260811 12:46:58.360530 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:46:58.360842 16405 heartbeater.cc:347] Connected to a master server at 127.7.111.126:46155
I20260811 12:46:58.361073 7613 external_mini_cluster.cc:1428] Setting key c89c74d2d262649c3072a0e269a890ff
I20260811 12:46:58.361152 16405 heartbeater.cc:464] Registering TS with master...
I20260811 12:46:58.361907 16405 heartbeater.cc:511] Master 127.7.111.126:46155 requested a full tablet report, sending...
I20260811 12:46:58.363410 15688 ts_manager.cc:194] Registered new tserver with Master: 1a1fe9b4b9d4467c8d13acb79a3ec8f9 (127.7.111.67:36221)
I20260811 12:46:58.364945 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:46:58.371551 15979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.369153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44398 (local address 127.7.111.126:46155)
0811 12:46:58.369304 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:58.369308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:58.369839 (+ 531us) server_negotiation.cc:407] Connection header received
0811 12:46:58.369990 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:58.369996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:58.370046 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:58.370125 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:58.370963 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:58.370969 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:58.370973 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:46:58.370991 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:58.371021 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:46:58.371032 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:58.371136 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:58.371251 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 12:46:58.371305 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:46:58.375056 15689 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44398:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:46:58.381716 15689 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:46:58.408509 16425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.392832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:36221 (local address 127.0.0.1:56320)
0811 12:46:58.393593 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:58.393634 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:58.393787 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:58.405934 (+ 12147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:58.405984 (+ 50us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:58.406079 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:58.406857 (+ 778us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:46:58.406861 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:58.406917 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:46:58.406999 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:58.407456 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:58.407467 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:58.407479 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 12:46:58.407655 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 12:46:58.407756 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":263,"threads_started":1}
I20260811 12:46:58.408736 16426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.392973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56320 (local address 127.7.111.67:36221)
0811 12:46:58.405188 (+ 12215us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:58.405198 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:58.405233 (+ 35us) server_negotiation.cc:407] Connection header received
0811 12:46:58.405306 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:58.405313 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:58.405591 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:58.405749 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:58.407205 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:58.407214 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:58.407229 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:46:58.407247 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:58.407307 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:46:58.407327 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:58.408164 (+ 837us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:58.408244 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 12:46:58.408353 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12011,"thread_start_us":11844,"threads_started":1}
I20260811 12:46:58.409350 16430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.394012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:40063 (local address 127.0.0.1:53628)
0811 12:46:58.403996 (+ 9984us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:58.404017 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:58.404131 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:58.407438 (+ 3307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:58.407442 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:58.407464 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:58.408172 (+ 708us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:46:58.408175 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:58.408192 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:46:58.408236 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:58.408718 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:58.408722 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:58.408727 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 12:46:58.408853 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 12:46:58.408901 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9840,"thread_start_us":11666,"threads_started":1}
I20260811 12:46:58.409950 16431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.394536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53628 (local address 127.7.111.65:40063)
0811 12:46:58.406213 (+ 11677us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:58.406223 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:58.406254 (+ 31us) server_negotiation.cc:407] Connection header received
0811 12:46:58.406322 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:58.406330 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:58.406539 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:58.406672 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:58.408408 (+ 1736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:58.408416 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:58.408430 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:46:58.408447 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:58.408501 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:46:58.408525 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:58.408648 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:58.409540 (+ 892us) server_negotiation.cc:299] Negotiation successful
0811 12:46:58.409640 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10776,"thread_start_us":10631,"threads_started":1}
I20260811 12:46:58.410825 16432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.394551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41204 (local address 127.7.111.66:44183)
0811 12:46:58.407321 (+ 12770us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:58.407329 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:58.407355 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:46:58.407422 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:58.407429 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:58.407636 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:58.407785 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:58.410021 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:58.410029 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:58.410042 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:46:58.410069 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:58.410132 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:46:58.410154 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:58.410327 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:58.410402 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 12:46:58.410491 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12607,"thread_start_us":12022,"threads_started":1}
I20260811 12:46:58.410835 16427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.394227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:44183 (local address 127.0.0.1:41204)
0811 12:46:58.405884 (+ 11657us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:58.405902 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:58.405996 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:58.408332 (+ 2336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:58.408337 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:58.408362 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:58.409359 (+ 997us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:46:58.409363 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:58.409411 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:46:58.409856 (+ 445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:58.410272 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:58.410277 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:58.410281 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 12:46:58.410409 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 12:46:58.410455 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11538,"thread_start_us":9439,"threads_started":1}
I20260811 12:46:58.413023 16302 tablet_service.cc:1467] Processing CreateTablet for tablet eaa3a63098ce435a99dfcd263a64891a (DEFAULT_TABLE table=test-table [id=bd5315abc5aa496cbb96d8525dc04886]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:58.413205 15880 tablet_service.cc:1467] Processing CreateTablet for tablet eaa3a63098ce435a99dfcd263a64891a (DEFAULT_TABLE table=test-table [id=bd5315abc5aa496cbb96d8525dc04886]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:58.414453 16302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaa3a63098ce435a99dfcd263a64891a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:58.414575 15880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaa3a63098ce435a99dfcd263a64891a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:58.414649 16077 tablet_service.cc:1467] Processing CreateTablet for tablet eaa3a63098ce435a99dfcd263a64891a (DEFAULT_TABLE table=test-table [id=bd5315abc5aa496cbb96d8525dc04886]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:46:58.415916 16077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaa3a63098ce435a99dfcd263a64891a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:58.425546 16433 tablet_bootstrap.cc:492] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9: Bootstrap starting.
I20260811 12:46:58.428840 16434 tablet_bootstrap.cc:492] T eaa3a63098ce435a99dfcd263a64891a P 7101e1c87f474d989066320831d49e1e: Bootstrap starting.
I20260811 12:46:58.429569 16433 tablet_bootstrap.cc:654] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:58.430300 16435 tablet_bootstrap.cc:492] T eaa3a63098ce435a99dfcd263a64891a P 7f123c21682f466c997ddd184969f5f8: Bootstrap starting.
I20260811 12:46:58.431295 16433 log.cc:824] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9: Log is configured to *not* fsync() on all Append() calls
I20260811 12:46:58.432271 16434 tablet_bootstrap.cc:654] T eaa3a63098ce435a99dfcd263a64891a P 7101e1c87f474d989066320831d49e1e: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:58.433429 16435 tablet_bootstrap.cc:654] T eaa3a63098ce435a99dfcd263a64891a P 7f123c21682f466c997ddd184969f5f8: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:58.433543 16434 log.cc:824] T eaa3a63098ce435a99dfcd263a64891a P 7101e1c87f474d989066320831d49e1e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:46:58.434607 16433 tablet_bootstrap.cc:492] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9: No bootstrap required, opened a new log
I20260811 12:46:58.434702 16435 log.cc:824] T eaa3a63098ce435a99dfcd263a64891a P 7f123c21682f466c997ddd184969f5f8: Log is configured to *not* fsync() on all Append() calls
I20260811 12:46:58.434891 16433 ts_tablet_manager.cc:1397] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260811 12:46:58.436215 16434 tablet_bootstrap.cc:492] T eaa3a63098ce435a99dfcd263a64891a P 7101e1c87f474d989066320831d49e1e: No bootstrap required, opened a new log
I20260811 12:46:58.436457 16434 ts_tablet_manager.cc:1397] T eaa3a63098ce435a99dfcd263a64891a P 7101e1c87f474d989066320831d49e1e: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 12:46:58.437939 16435 tablet_bootstrap.cc:492] T eaa3a63098ce435a99dfcd263a64891a P 7f123c21682f466c997ddd184969f5f8: No bootstrap required, opened a new log
I20260811 12:46:58.438185 16435 ts_tablet_manager.cc:1397] T eaa3a63098ce435a99dfcd263a64891a P 7f123c21682f466c997ddd184969f5f8: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260811 12:46:58.443821 16434 raft_consensus.cc:348] T eaa3a63098ce435a99dfcd263a64891a P 7101e1c87f474d989066320831d49e1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1fe9b4b9d4467c8d13acb79a3ec8f9" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 36221 } } peers { permanent_uuid: "7101e1c87f474d989066320831d49e1e" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40063 } } peers { permanent_uuid: "7f123c21682f466c997ddd184969f5f8" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 44183 } }
I20260811 12:46:58.444176 16434 raft_consensus.cc:374] T eaa3a63098ce435a99dfcd263a64891a P 7101e1c87f474d989066320831d49e1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:58.444310 16434 raft_consensus.cc:729] T eaa3a63098ce435a99dfcd263a64891a P 7101e1c87f474d989066320831d49e1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7101e1c87f474d989066320831d49e1e, State: Initialized, Role: FOLLOWER
I20260811 12:46:58.444871 16434 consensus_queue.cc:260] T eaa3a63098ce435a99dfcd263a64891a P 7101e1c87f474d989066320831d49e1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1fe9b4b9d4467c8d13acb79a3ec8f9" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 36221 } } peers { permanent_uuid: "7101e1c87f474d989066320831d49e1e" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40063 } } peers { permanent_uuid: "7f123c21682f466c997ddd184969f5f8" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 44183 } }
I20260811 12:46:58.444945 16433 raft_consensus.cc:348] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1fe9b4b9d4467c8d13acb79a3ec8f9" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 36221 } } peers { permanent_uuid: "7101e1c87f474d989066320831d49e1e" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40063 } } peers { permanent_uuid: "7f123c21682f466c997ddd184969f5f8" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 44183 } }
I20260811 12:46:58.445461 16433 raft_consensus.cc:374] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:58.445598 16433 raft_consensus.cc:729] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a1fe9b4b9d4467c8d13acb79a3ec8f9, State: Initialized, Role: FOLLOWER
I20260811 12:46:58.445935 15977 heartbeater.cc:503] Master 127.7.111.126:46155 was elected leader, sending a full tablet report...
I20260811 12:46:58.446205 16433 consensus_queue.cc:260] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1fe9b4b9d4467c8d13acb79a3ec8f9" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 36221 } } peers { permanent_uuid: "7101e1c87f474d989066320831d49e1e" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40063 } } peers { permanent_uuid: "7f123c21682f466c997ddd184969f5f8" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 44183 } }
I20260811 12:46:58.446697 16434 ts_tablet_manager.cc:1428] T eaa3a63098ce435a99dfcd263a64891a P 7101e1c87f474d989066320831d49e1e: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260811 12:46:58.447695 16405 heartbeater.cc:503] Master 127.7.111.126:46155 was elected leader, sending a full tablet report...
I20260811 12:46:58.448351 16433 ts_tablet_manager.cc:1428] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260811 12:46:58.448248 16435 raft_consensus.cc:348] T eaa3a63098ce435a99dfcd263a64891a P 7f123c21682f466c997ddd184969f5f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1fe9b4b9d4467c8d13acb79a3ec8f9" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 36221 } } peers { permanent_uuid: "7101e1c87f474d989066320831d49e1e" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40063 } } peers { permanent_uuid: "7f123c21682f466c997ddd184969f5f8" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 44183 } }
I20260811 12:46:58.448654 16435 raft_consensus.cc:374] T eaa3a63098ce435a99dfcd263a64891a P 7f123c21682f466c997ddd184969f5f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:58.448760 16435 raft_consensus.cc:729] T eaa3a63098ce435a99dfcd263a64891a P 7f123c21682f466c997ddd184969f5f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f123c21682f466c997ddd184969f5f8, State: Initialized, Role: FOLLOWER
I20260811 12:46:58.449734 16435 consensus_queue.cc:260] T eaa3a63098ce435a99dfcd263a64891a P 7f123c21682f466c997ddd184969f5f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1fe9b4b9d4467c8d13acb79a3ec8f9" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 36221 } } peers { permanent_uuid: "7101e1c87f474d989066320831d49e1e" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40063 } } peers { permanent_uuid: "7f123c21682f466c997ddd184969f5f8" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 44183 } }
I20260811 12:46:58.450853 16187 heartbeater.cc:503] Master 127.7.111.126:46155 was elected leader, sending a full tablet report...
I20260811 12:46:58.451485 16435 ts_tablet_manager.cc:1428] T eaa3a63098ce435a99dfcd263a64891a P 7f123c21682f466c997ddd184969f5f8: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
W20260811 12:46:58.463680 15978 tablet.cc:2369] T eaa3a63098ce435a99dfcd263a64891a P 7101e1c87f474d989066320831d49e1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:46:58.529436 16441 raft_consensus.cc:482] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:46:58.529760 16441 raft_consensus.cc:504] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1fe9b4b9d4467c8d13acb79a3ec8f9" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 36221 } } peers { permanent_uuid: "7101e1c87f474d989066320831d49e1e" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40063 } } peers { permanent_uuid: "7f123c21682f466c997ddd184969f5f8" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 44183 } }
I20260811 12:46:58.531608 16441 leader_election.cc:288] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7101e1c87f474d989066320831d49e1e (127.7.111.65:40063), 7f123c21682f466c997ddd184969f5f8 (127.7.111.66:44183)
I20260811 12:46:58.534552 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.531970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:44183 (local address 127.7.111.67:42259)
0811 12:46:58.532152 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:58.532172 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:58.532290 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:58.533019 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:58.533023 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:58.533058 (+ 35us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:58.533719 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:46:58.533724 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:58.533740 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:46:58.533787 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:58.534154 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:58.534159 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:58.534165 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 12:46:58.534260 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 12:46:58.534310 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 12:46:58.534583 16432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.532370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:42259 (local address 127.7.111.66:44183)
0811 12:46:58.532573 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:58.532578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:58.532596 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:46:58.532659 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:58.532663 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:58.532722 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:58.532822 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:58.533952 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:58.533959 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:58.533964 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:46:58.533985 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:58.534021 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:46:58.534034 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:58.534167 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:58.534286 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 12:46:58.534353 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:46:58.535161 16431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.532228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:55813 (local address 127.7.111.65:40063)
0811 12:46:58.532416 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:58.532422 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:58.533261 (+ 839us) server_negotiation.cc:407] Connection header received
0811 12:46:58.533362 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:58.533365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:58.533447 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:58.533521 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:58.534470 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:58.534475 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:58.534478 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:46:58.534498 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:58.534535 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:46:58.534552 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:58.534671 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:58.534858 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 12:46:58.534927 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 12:46:58.535727 16444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.531978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:40063 (local address 127.7.111.67:55813)
0811 12:46:58.533144 (+ 1166us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:58.533162 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:58.533261 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:58.533567 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:58.533572 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:58.533597 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:46:58.534202 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:46:58.534206 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:46:58.534223 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:46:58.534271 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:46:58.534685 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:46:58.534691 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:46:58.534696 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 12:46:58.535374 (+ 678us) client_negotiation.cc:240] Negotiation successful
0811 12:46:58.535416 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1055,"mutex_wait_us":457,"thread_start_us":424,"threads_started":1}
I20260811 12:46:58.535558 16112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaa3a63098ce435a99dfcd263a64891a" candidate_uuid: "1a1fe9b4b9d4467c8d13acb79a3ec8f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f123c21682f466c997ddd184969f5f8" is_pre_election: true
I20260811 12:46:58.536051 16112 raft_consensus.cc:2454] T eaa3a63098ce435a99dfcd263a64891a P 7f123c21682f466c997ddd184969f5f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a1fe9b4b9d4467c8d13acb79a3ec8f9 in term 0.
I20260811 12:46:58.536661 16220 leader_election.cc:302] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a1fe9b4b9d4467c8d13acb79a3ec8f9, 7f123c21682f466c997ddd184969f5f8; no voters:
I20260811 12:46:58.536659 15912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaa3a63098ce435a99dfcd263a64891a" candidate_uuid: "1a1fe9b4b9d4467c8d13acb79a3ec8f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7101e1c87f474d989066320831d49e1e" is_pre_election: true
I20260811 12:46:58.537004 16441 raft_consensus.cc:2781] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:46:58.537128 16441 raft_consensus.cc:482] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:46:58.537115 15912 raft_consensus.cc:2454] T eaa3a63098ce435a99dfcd263a64891a P 7101e1c87f474d989066320831d49e1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a1fe9b4b9d4467c8d13acb79a3ec8f9 in term 0.
I20260811 12:46:58.537228 16441 raft_consensus.cc:3037] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:58.539433 16441 raft_consensus.cc:504] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1fe9b4b9d4467c8d13acb79a3ec8f9" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 36221 } } peers { permanent_uuid: "7101e1c87f474d989066320831d49e1e" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40063 } } peers { permanent_uuid: "7f123c21682f466c997ddd184969f5f8" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 44183 } }
I20260811 12:46:58.540017 16441 leader_election.cc:288] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9 [CANDIDATE]: Term 1 election: Requested vote from peers 7101e1c87f474d989066320831d49e1e (127.7.111.65:40063), 7f123c21682f466c997ddd184969f5f8 (127.7.111.66:44183)
I20260811 12:46:58.540372 15912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaa3a63098ce435a99dfcd263a64891a" candidate_uuid: "1a1fe9b4b9d4467c8d13acb79a3ec8f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7101e1c87f474d989066320831d49e1e"
I20260811 12:46:58.540498 16112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaa3a63098ce435a99dfcd263a64891a" candidate_uuid: "1a1fe9b4b9d4467c8d13acb79a3ec8f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f123c21682f466c997ddd184969f5f8"
I20260811 12:46:58.540577 15912 raft_consensus.cc:3037] T eaa3a63098ce435a99dfcd263a64891a P 7101e1c87f474d989066320831d49e1e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:58.540716 16112 raft_consensus.cc:3037] T eaa3a63098ce435a99dfcd263a64891a P 7f123c21682f466c997ddd184969f5f8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:58.543833 16112 raft_consensus.cc:2454] T eaa3a63098ce435a99dfcd263a64891a P 7f123c21682f466c997ddd184969f5f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a1fe9b4b9d4467c8d13acb79a3ec8f9 in term 1.
I20260811 12:46:58.543833 15912 raft_consensus.cc:2454] T eaa3a63098ce435a99dfcd263a64891a P 7101e1c87f474d989066320831d49e1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a1fe9b4b9d4467c8d13acb79a3ec8f9 in term 1.
I20260811 12:46:58.544222 16220 leader_election.cc:302] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a1fe9b4b9d4467c8d13acb79a3ec8f9, 7f123c21682f466c997ddd184969f5f8; no voters:
I20260811 12:46:58.544499 16441 raft_consensus.cc:2781] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:46:58.544919 16441 raft_consensus.cc:686] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9 [term 1 LEADER]: Becoming Leader. State: Replica: 1a1fe9b4b9d4467c8d13acb79a3ec8f9, State: Running, Role: LEADER
I20260811 12:46:58.545449 16441 consensus_queue.cc:237] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1fe9b4b9d4467c8d13acb79a3ec8f9" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 36221 } } peers { permanent_uuid: "7101e1c87f474d989066320831d49e1e" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40063 } } peers { permanent_uuid: "7f123c21682f466c997ddd184969f5f8" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 44183 } }
I20260811 12:46:58.550017 15684 catalog_manager.cc:5495] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a1fe9b4b9d4467c8d13acb79a3ec8f9 (127.7.111.67). New cstate: current_term: 1 leader_uuid: "1a1fe9b4b9d4467c8d13acb79a3ec8f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1fe9b4b9d4467c8d13acb79a3ec8f9" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 36221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7101e1c87f474d989066320831d49e1e" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f123c21682f466c997ddd184969f5f8" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 44183 } health_report { overall_health: UNKNOWN } } }
I20260811 12:46:58.570820 16426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.568999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56328 (local address 127.7.111.67:36221)
0811 12:46:58.569193 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:58.569198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:58.569215 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:46:58.569360 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:58.569363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:58.569477 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:58.569560 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:58.570325 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:58.570329 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:58.570332 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:46:58.570343 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:58.570371 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:46:58.570381 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:58.570474 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:58.570536 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 12:46:58.570582 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 12:46:58.579350 16112 raft_consensus.cc:1261] T eaa3a63098ce435a99dfcd263a64891a P 7f123c21682f466c997ddd184969f5f8 [term 1 FOLLOWER]: Refusing update from remote peer 1a1fe9b4b9d4467c8d13acb79a3ec8f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:46:58.579352 15912 raft_consensus.cc:1261] T eaa3a63098ce435a99dfcd263a64891a P 7101e1c87f474d989066320831d49e1e [term 1 FOLLOWER]: Refusing update from remote peer 1a1fe9b4b9d4467c8d13acb79a3ec8f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:46:58.580430 16445 consensus_queue.cc:1035] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7101e1c87f474d989066320831d49e1e" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:46:58.580804 16441 consensus_queue.cc:1035] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f123c21682f466c997ddd184969f5f8" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 44183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:46:58.608525 15689 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:46:58.608793 15689 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:46:58.612429 15689 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fd5df18c20da4303bac88b841cd6c5db: Sending DeleteTablet for 3 replicas of tablet eaa3a63098ce435a99dfcd263a64891a
I20260811 12:46:58.613526 16302 tablet_service.cc:1514] Processing DeleteTablet for tablet eaa3a63098ce435a99dfcd263a64891a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:58 UTC) from {username='slave'} at 127.0.0.1:56320
I20260811 12:46:58.613667 15880 tablet_service.cc:1514] Processing DeleteTablet for tablet eaa3a63098ce435a99dfcd263a64891a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:58 UTC) from {username='slave'} at 127.0.0.1:53628
I20260811 12:46:58.613667 16077 tablet_service.cc:1514] Processing DeleteTablet for tablet eaa3a63098ce435a99dfcd263a64891a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:46:58 UTC) from {username='slave'} at 127.0.0.1:41204
I20260811 12:46:58.617825 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 15786
I20260811 12:46:58.624643 16465 tablet_replica.cc:330] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9: stopping tablet replica
I20260811 12:46:58.625334 16465 raft_consensus.cc:2229] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:46:58.626722 16465 raft_consensus.cc:2258] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:46:58.628001 16431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.619708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53644 (local address 127.7.111.65:40063)
0811 12:46:58.619906 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:58.619913 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:58.623951 (+ 4038us) server_negotiation.cc:407] Connection header received
0811 12:46:58.624140 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:58.624147 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:58.624296 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:58.624409 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:58.626905 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:58.626912 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:58.626917 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:46:58.626942 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:58.626989 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:46:58.627007 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:58.627115 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:58.627554 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 12:46:58.627632 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:46:58.629264 16465 ts_tablet_manager.cc:1905] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:46:58.629354 15971 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:58.636590 16467 tablet_replica.cc:330] T eaa3a63098ce435a99dfcd263a64891a P 7f123c21682f466c997ddd184969f5f8: stopping tablet replica
I20260811 12:46:58.637230 16465 ts_tablet_manager.cc:1918] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:46:58.637408 16467 raft_consensus.cc:2229] T eaa3a63098ce435a99dfcd263a64891a P 7f123c21682f466c997ddd184969f5f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:46:58.637560 16465 log.cc:1196] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/eaa3a63098ce435a99dfcd263a64891a
I20260811 12:46:58.638101 16467 raft_consensus.cc:2258] T eaa3a63098ce435a99dfcd263a64891a P 7f123c21682f466c997ddd184969f5f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:46:58.638278 16465 ts_tablet_manager.cc:1939] T eaa3a63098ce435a99dfcd263a64891a P 1a1fe9b4b9d4467c8d13acb79a3ec8f9: Deleting consensus metadata
I20260811 12:46:58.640468 15667 catalog_manager.cc:4841] TS 1a1fe9b4b9d4467c8d13acb79a3ec8f9 (127.7.111.67:36221): tablet eaa3a63098ce435a99dfcd263a64891a (table test-table [id=bd5315abc5aa496cbb96d8525dc04886]) successfully deleted
I20260811 12:46:58.640805 16467 ts_tablet_manager.cc:1905] T eaa3a63098ce435a99dfcd263a64891a P 7f123c21682f466c997ddd184969f5f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:46:58.648729 16467 ts_tablet_manager.cc:1918] T eaa3a63098ce435a99dfcd263a64891a P 7f123c21682f466c997ddd184969f5f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:46:58.648890 16467 log.cc:1196] T eaa3a63098ce435a99dfcd263a64891a P 7f123c21682f466c997ddd184969f5f8: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/eaa3a63098ce435a99dfcd263a64891a
I20260811 12:46:58.649320 16467 ts_tablet_manager.cc:1939] T eaa3a63098ce435a99dfcd263a64891a P 7f123c21682f466c997ddd184969f5f8: Deleting consensus metadata
I20260811 12:46:58.650974 15669 catalog_manager.cc:4841] TS 7f123c21682f466c997ddd184969f5f8 (127.7.111.66:44183): tablet eaa3a63098ce435a99dfcd263a64891a (table test-table [id=bd5315abc5aa496cbb96d8525dc04886]) successfully deleted
I20260811 12:46:58.749419 16466 tablet_replica.cc:330] T eaa3a63098ce435a99dfcd263a64891a P 7101e1c87f474d989066320831d49e1e: stopping tablet replica
I20260811 12:46:58.749718 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 15786
W20260811 12:46:58.777632 15667 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.111.65:40063 (error 108)
I20260811 12:46:58.777807 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 15981
I20260811 12:46:58.779815 16427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.779292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:40063 (local address 127.0.0.1:53650)
0811 12:46:58.779496 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:58.779642 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:40063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 12:46:58.780136 15667 catalog_manager.cc:4584] TS 7101e1c87f474d989066320831d49e1e (127.7.111.65:40063): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eaa3a63098ce435a99dfcd263a64891a: Network error: Client connection negotiation failed: client connection to 127.7.111.65:40063: connect: Connection refused (error 111)
I20260811 12:46:58.780982 16432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.778867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41212 (local address 127.7.111.66:44183)
0811 12:46:58.779037 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:58.779042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:58.779059 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:46:58.779230 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:58.779236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:58.779356 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:58.779467 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:58.780358 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:58.780362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:58.780365 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:46:58.780382 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:58.780421 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:46:58.780437 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:58.780554 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:58.780668 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 12:46:58.780732 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 12:46:58.781847 16178 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:58.834193 16427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.833626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:40063 (local address 127.0.0.1:53654)
0811 12:46:58.833865 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:58.833993 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:40063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 12:46:58.835379 16427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.834967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:40063 (local address 127.0.0.1:53668)
0811 12:46:58.835135 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:58.835228 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:40063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 12:46:58.881424 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 15981
I20260811 12:46:58.896085 16427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.895492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:40063 (local address 127.0.0.1:53678)
0811 12:46:58.895745 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:58.895870 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:40063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 12:46:58.897285 16427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.896878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:40063 (local address 127.0.0.1:53690)
0811 12:46:58.897047 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:58.897135 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:40063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 12:46:58.910821 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 16194
I20260811 12:46:58.913968 16426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.911649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56334 (local address 127.7.111.67:36221)
0811 12:46:58.911855 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:58.911861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:58.911911 (+ 50us) server_negotiation.cc:407] Connection header received
0811 12:46:58.912002 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:58.912007 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:58.912077 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:58.912208 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:58.913132 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:58.913138 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:58.913143 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:46:58.913164 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:58.913204 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:46:58.913220 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:58.913412 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:58.913533 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 12:46:58.913601 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 12:46:58.914714 16395 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:58.978284 16427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.977765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:40063 (local address 127.0.0.1:53698)
0811 12:46:58.978006 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:58.978113 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:40063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 12:46:58.979444 16427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:58.979022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:40063 (local address 127.0.0.1:53700)
0811 12:46:58.979257 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:58.979327 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:40063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260811 12:46:59.035815 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 16194
I20260811 12:46:59.063702 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 15631
I20260811 12:46:59.064818 15742 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:59.172026 16427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:59.171372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:40063 (local address 127.0.0.1:53708)
0811 12:46:59.171594 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:59.171728 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:40063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 12:46:59.172286 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 15631
2026-08-11T12:46:59Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2743 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 12:46:59.220551 7613 mini_oidc.cc:152] Starting JWKS server
I20260811 12:46:59.223325 7613 webserver.cc:460] Webserver started at https://127.0.0.1:38065/ using document root <none> and password file <none>
I20260811 12:46:59.223642 7613 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 12:46:59.224237 7613 webserver.cc:460] Webserver started at http://127.0.0.1:45233/ using document root <none> and password file <none>
2026-08-11T12:46:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:46:59Z Disabled control of system clock
I20260811 12:46:59.257401 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:34919
--webserver_interface=127.7.111.126
--webserver_port=0
--builtin_ntp_servers=127.7.111.84:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:34919
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38065/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786452375851340-7613-0/testchainca.cert with env {}
W20260811 12:46:59.585292 16484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:46:59.585647 16484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:46:59.585786 16484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:46:59.593433 16484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:46:59.593560 16484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:46:59.593626 16484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:46:59.593708 16484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:46:59.593770 16484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 12:46:59.593855 16484 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 12:46:59.593909 16484 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38065/jwks.json
I20260811 12:46:59.604939 16484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:34919
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:34919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38065/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786452375851340-7613-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:46:59.606372 16484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:46:59.607702 16484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:46:59.619274 16494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:59.619318 16499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:46:59.619499 16492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:46:59.620280 16484 server_base.cc:1034] running on GCE node
I20260811 12:46:59.621083 16484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:46:59.622537 16484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:46:59.623744 16484 hybrid_clock.cc:648] HybridClock initialized: now 1786452419623685 us; error 54 us; skew 500 ppm
I20260811 12:46:59.624092 16484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:46:59.626384 16484 webserver.cc:460] Webserver started at http://127.7.111.126:40283/ using document root <none> and password file <none>
I20260811 12:46:59.627080 16484 fs_manager.cc:359] Metadata directory not provided
I20260811 12:46:59.627200 16484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:46:59.627506 16484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:46:59.630503 16484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "d6a38936876849869e7da36dabda56b8"
format_stamp: "Formatted at 2026-08-11 12:46:59 on dist-test-slave-f0tn"
server_key: "a540358ecd3bac2ce627bb4b786b8638"
server_key_iv: "f8547741b4410d565ba83ca015809fc7"
server_key_version: "encryptionkey@0"
I20260811 12:46:59.631261 16484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "d6a38936876849869e7da36dabda56b8"
format_stamp: "Formatted at 2026-08-11 12:46:59 on dist-test-slave-f0tn"
server_key: "a540358ecd3bac2ce627bb4b786b8638"
server_key_iv: "f8547741b4410d565ba83ca015809fc7"
server_key_version: "encryptionkey@0"
I20260811 12:46:59.636233 16484 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 12:46:59.639657 16507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:46:59.641063 16484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:46:59.641248 16484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "d6a38936876849869e7da36dabda56b8"
format_stamp: "Formatted at 2026-08-11 12:46:59 on dist-test-slave-f0tn"
server_key: "a540358ecd3bac2ce627bb4b786b8638"
server_key_iv: "f8547741b4410d565ba83ca015809fc7"
server_key_version: "encryptionkey@0"
I20260811 12:46:59.641498 16484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:46:59.665040 16484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:46:59.665889 16484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:46:59.666203 16484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:46:59.683259 16484 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:34919
I20260811 12:46:59.683277 16580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:34919 every 8 connection(s)
I20260811 12:46:59.684518 16484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:46:59.687860 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 16484
I20260811 12:46:59.688090 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:46:59.688320 16582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:46:59.688915 7613 external_mini_cluster.cc:1428] Setting key 8f6a1fa4e7118606cc0d91615241ac12
I20260811 12:46:59.701262 16582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6a38936876849869e7da36dabda56b8: Bootstrap starting.
I20260811 12:46:59.702960 16587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:59.690835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33152 (local address 127.7.111.126:34919)
0811 12:46:59.691811 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:59.691821 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:59.691863 (+ 42us) server_negotiation.cc:407] Connection header received
0811 12:46:59.694224 (+ 2361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:59.694248 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:59.694702 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:59.695020 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:46:59.695665 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:59.696886 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:59.698754 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:59.699407 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:59.699541 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:59.700762 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:46:59.700770 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:46:59.700783 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:46:59.701290 (+ 507us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:46:59.701366 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:46:59.701410 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:46:59.701522 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 12:46:59.701715 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 12:46:59.702031 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":352,"threads_started":1}
I20260811 12:46:59.704736 16582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6a38936876849869e7da36dabda56b8: Neither blocks nor log segments found. Creating new log.
I20260811 12:46:59.705879 16582 log.cc:824] T 00000000000000000000000000000000 P d6a38936876849869e7da36dabda56b8: Log is configured to *not* fsync() on all Append() calls
I20260811 12:46:59.706758 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:46:59.708606 16582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6a38936876849869e7da36dabda56b8: No bootstrap required, opened a new log
I20260811 12:46:59.715946 16582 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6a38936876849869e7da36dabda56b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6a38936876849869e7da36dabda56b8" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34919 } }
I20260811 12:46:59.716269 16582 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6a38936876849869e7da36dabda56b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:46:59.716362 16582 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6a38936876849869e7da36dabda56b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a38936876849869e7da36dabda56b8, State: Initialized, Role: FOLLOWER
I20260811 12:46:59.716851 16582 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6a38936876849869e7da36dabda56b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6a38936876849869e7da36dabda56b8" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34919 } }
I20260811 12:46:59.717057 16582 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6a38936876849869e7da36dabda56b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:46:59.717180 16582 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6a38936876849869e7da36dabda56b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:46:59.717340 16582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6a38936876849869e7da36dabda56b8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:46:59.719813 16582 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6a38936876849869e7da36dabda56b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6a38936876849869e7da36dabda56b8" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34919 } }
I20260811 12:46:59.720255 16582 leader_election.cc:302] T 00000000000000000000000000000000 P d6a38936876849869e7da36dabda56b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6a38936876849869e7da36dabda56b8; no voters:
I20260811 12:46:59.720794 16582 leader_election.cc:288] T 00000000000000000000000000000000 P d6a38936876849869e7da36dabda56b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:46:59.720932 16589 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6a38936876849869e7da36dabda56b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:46:59.721642 16589 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6a38936876849869e7da36dabda56b8 [term 1 LEADER]: Becoming Leader. State: Replica: d6a38936876849869e7da36dabda56b8, State: Running, Role: LEADER
I20260811 12:46:59.722148 16589 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6a38936876849869e7da36dabda56b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6a38936876849869e7da36dabda56b8" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34919 } }
I20260811 12:46:59.722711 16582 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6a38936876849869e7da36dabda56b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:46:59.724864 16592 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6a38936876849869e7da36dabda56b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6a38936876849869e7da36dabda56b8. Latest consensus state: current_term: 1 leader_uuid: "d6a38936876849869e7da36dabda56b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6a38936876849869e7da36dabda56b8" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34919 } } }
I20260811 12:46:59.725099 16590 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6a38936876849869e7da36dabda56b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6a38936876849869e7da36dabda56b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6a38936876849869e7da36dabda56b8" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34919 } } }
I20260811 12:46:59.725939 16590 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6a38936876849869e7da36dabda56b8 [sys.catalog]: This master's current role is: LEADER
I20260811 12:46:59.726547 16592 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6a38936876849869e7da36dabda56b8 [sys.catalog]: This master's current role is: LEADER
I20260811 12:46:59.727979 16605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:46:59.733220 16605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:46:59.741336 16605 catalog_manager.cc:1347] Generated new cluster ID: d170e43218e74fa893db455144c879b5
I20260811 12:46:59.741489 16605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:46:59.769166 16605 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:46:59.770462 16605 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:46:59.779843 16605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6a38936876849869e7da36dabda56b8: Generated new TSK 0
I20260811 12:46:59.780678 16605 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 12:46:59.827925 16631 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 12:46:59.828122 16631 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 12:46:59.830506 16587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:46:59.806341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33162 (local address 127.7.111.126:34919)
0811 12:46:59.806577 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 12:46:59.806585 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:46:59.806952 (+ 367us) server_negotiation.cc:407] Connection header received
0811 12:46:59.807293 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:46:59.807301 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:46:59.807406 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:46:59.807582 (+ 176us) server_negotiation.cc:226] Negotiated authn=JWT
0811 12:46:59.808076 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:59.809190 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:59.810650 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:46:59.811344 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:46:59.811488 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:59.811992 (+ 504us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 12:46:59.829790 (+ 17798us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38065/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 12:46:59.830175 (+ 385us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33162: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38065/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":79}
W20260811 12:46:59.830646 16587 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33162: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38065/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 12:46:59.831928 16630 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:46:59.806182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34919 (local address 127.0.0.1:33162)
0811 12:46:59.806852 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0811 12:46:59.806868 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:46:59.807014 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:46:59.807600 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:46:59.807604 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:46:59.807614 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0811 12:46:59.807923 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:59.807931 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:59.809322 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:59.809325 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:59.810506 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:46:59.810518 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:46:59.811520 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:46:59.811524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:46:59.811646 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:46:59.811709 (+ 63us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 12:46:59.830065 (+ 18356us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38065/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 12:46:59.830351 (+ 286us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.111.126:34919: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38065/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":317,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.7.111.126:34919: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38065/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 12:46:59.833793 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 16484
I20260811 12:46:59.834966 16575 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:46:59.928706 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 16484
2026-08-11T12:46:59Z chronyd exiting
I20260811 12:46:59.982718 7613 test_util.cc:175] -----------------------------------------------
I20260811 12:46:59.982856 7613 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786452375851340-7613-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (766 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 12:46:59.987254 7613 mini_oidc.cc:152] Starting JWKS server
I20260811 12:46:59.989112 7613 webserver.cc:460] Webserver started at https://127.0.0.1:41985/ using document root <none> and password file <none>
I20260811 12:46:59.989332 7613 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 12:46:59.989996 7613 webserver.cc:460] Webserver started at http://127.0.0.1:45791/ using document root <none> and password file <none>
2026-08-11T12:47:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:47:00Z Disabled control of system clock
I20260811 12:47:00.025239 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:46771
--webserver_interface=127.7.111.126
--webserver_port=0
--builtin_ntp_servers=127.7.111.84:41653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:46771
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41985/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786452375851340-7613-0/testchainca.cert with env {}
W20260811 12:47:00.345775 16639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:00.346079 16639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:00.346208 16639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:00.353261 16639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:47:00.353394 16639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:00.353464 16639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:00.353543 16639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:47:00.353603 16639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 12:47:00.353690 16639 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 12:47:00.353749 16639 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41985/jwks.json
I20260811 12:47:00.363533 16639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:41653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:46771
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:46771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41985/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786452375851340-7613-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:00.364814 16639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:00.366132 16639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:00.377449 16650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:00.377408 16651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:00.377478 16654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:00.378242 16639 server_base.cc:1034] running on GCE node
I20260811 12:47:00.378978 16639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:00.380295 16639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:00.381466 16639 hybrid_clock.cc:648] HybridClock initialized: now 1786452420381412 us; error 40 us; skew 500 ppm
I20260811 12:47:00.381820 16639 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:47:00.383963 16639 webserver.cc:460] Webserver started at http://127.7.111.126:46455/ using document root <none> and password file <none>
I20260811 12:47:00.384631 16639 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:00.384737 16639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:00.385056 16639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:00.387938 16639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "f1e0e7363a6f4ce3aa515fffae78bfea"
format_stamp: "Formatted at 2026-08-11 12:47:00 on dist-test-slave-f0tn"
server_key: "1865d68d689919fb619e2a081a70a9ab"
server_key_iv: "e75eff94f9cdcf23d5976ddf1cad70c2"
server_key_version: "encryptionkey@0"
I20260811 12:47:00.388701 16639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "f1e0e7363a6f4ce3aa515fffae78bfea"
format_stamp: "Formatted at 2026-08-11 12:47:00 on dist-test-slave-f0tn"
server_key: "1865d68d689919fb619e2a081a70a9ab"
server_key_iv: "e75eff94f9cdcf23d5976ddf1cad70c2"
server_key_version: "encryptionkey@0"
I20260811 12:47:00.393849 16639 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:47:00.397189 16661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:00.398509 16639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:47:00.398669 16639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "f1e0e7363a6f4ce3aa515fffae78bfea"
format_stamp: "Formatted at 2026-08-11 12:47:00 on dist-test-slave-f0tn"
server_key: "1865d68d689919fb619e2a081a70a9ab"
server_key_iv: "e75eff94f9cdcf23d5976ddf1cad70c2"
server_key_version: "encryptionkey@0"
I20260811 12:47:00.398875 16639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:00.424499 16639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:00.425407 16639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:00.425709 16639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:00.443241 16639 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:46771
I20260811 12:47:00.443248 16753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:46771 every 8 connection(s)
I20260811 12:47:00.444597 16639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:47:00.448479 16754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:00.454929 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 16639
I20260811 12:47:00.455114 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:47:00.455766 7613 external_mini_cluster.cc:1428] Setting key 324ffca742b333d14bb40022305a8381
I20260811 12:47:00.460201 16754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea: Bootstrap starting.
I20260811 12:47:00.464622 16754 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:00.465924 16754 log.cc:824] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea: Log is configured to *not* fsync() on all Append() calls
I20260811 12:47:00.469195 16754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea: No bootstrap required, opened a new log
I20260811 12:47:00.471735 16760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:00.457876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49720 (local address 127.7.111.126:46771)
0811 12:47:00.459006 (+ 1130us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:00.459023 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:00.459081 (+ 58us) server_negotiation.cc:407] Connection header received
0811 12:47:00.461515 (+ 2434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:00.461545 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:00.462240 (+ 695us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:00.462731 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:00.463280 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:00.465288 (+ 2008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:00.466721 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:00.467545 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:00.467748 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:00.468815 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:00.468828 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:00.468847 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:47:00.469575 (+ 728us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:00.469694 (+ 119us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:47:00.469733 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:00.469867 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:00.470048 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 12:47:00.470370 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":317,"threads_started":1}
I20260811 12:47:00.475543 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:47:00.477772 16754 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e0e7363a6f4ce3aa515fffae78bfea" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46771 } }
I20260811 12:47:00.478119 16754 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:00.478229 16754 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e0e7363a6f4ce3aa515fffae78bfea, State: Initialized, Role: FOLLOWER
I20260811 12:47:00.479117 16754 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e0e7363a6f4ce3aa515fffae78bfea" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46771 } }
I20260811 12:47:00.479450 16762 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:47:00.479686 16762 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e0e7363a6f4ce3aa515fffae78bfea" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46771 } }
I20260811 12:47:00.479408 16754 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:47:00.480299 16754 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:47:00.480273 16762 leader_election.cc:302] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1e0e7363a6f4ce3aa515fffae78bfea; no voters:
I20260811 12:47:00.480468 16754 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:00.481132 16762 leader_election.cc:288] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260811 12:47:00.483007 16754 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e0e7363a6f4ce3aa515fffae78bfea" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46771 } }
I20260811 12:47:00.483223 16754 leader_election.cc:302] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1e0e7363a6f4ce3aa515fffae78bfea; no voters:
I20260811 12:47:00.483278 16763 raft_consensus.cc:2741] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260811 12:47:00.483423 16754 leader_election.cc:288] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:47:00.483753 16762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:47:00.484808 16762 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea [term 1 LEADER]: Becoming Leader. State: Replica: f1e0e7363a6f4ce3aa515fffae78bfea, State: Running, Role: LEADER
I20260811 12:47:00.485282 16762 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e0e7363a6f4ce3aa515fffae78bfea" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46771 } }
I20260811 12:47:00.485491 16754 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:47:00.487365 16763 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1e0e7363a6f4ce3aa515fffae78bfea. Latest consensus state: current_term: 1 leader_uuid: "f1e0e7363a6f4ce3aa515fffae78bfea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e0e7363a6f4ce3aa515fffae78bfea" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46771 } } }
I20260811 12:47:00.487821 16763 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:00.489281 16765 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1e0e7363a6f4ce3aa515fffae78bfea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e0e7363a6f4ce3aa515fffae78bfea" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46771 } } }
I20260811 12:47:00.490252 16765 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:00.491060 16779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:47:00.495561 16779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:47:00.503887 16779 catalog_manager.cc:1347] Generated new cluster ID: 7174eecd592445c8a0aa299bb81fd3ca
I20260811 12:47:00.504004 16779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:47:00.513548 16779 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:47:00.515313 16779 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:47:00.526633 16779 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1e0e7363a6f4ce3aa515fffae78bfea: Generated new TSK 0
I20260811 12:47:00.527616 16779 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 12:47:00.559702 16807 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 12:47:00.559841 16807 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 12:47:00.560844 16760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:00.541539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49726 (local address 127.7.111.126:46771)
0811 12:47:00.541767 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:00.541773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:00.542113 (+ 340us) server_negotiation.cc:407] Connection header received
0811 12:47:00.542355 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:00.542359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:00.542430 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:00.542551 (+ 121us) server_negotiation.cc:226] Negotiated authn=JWT
0811 12:47:00.543071 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:00.543768 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:00.545785 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:00.546217 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:00.546311 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:00.546867 (+ 556us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 12:47:00.560323 (+ 13456us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41985/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 12:47:00.560617 (+ 294us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49726: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41985/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 12:47:00.560969 16760 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49726: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41985/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 12:47:00.561297 16806 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:00.541349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:46771 (local address 127.0.0.1:49726)
0811 12:47:00.542010 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:00.542029 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:00.542180 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:00.542588 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:00.542594 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:00.542607 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0811 12:47:00.542905 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:00.542919 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:00.543904 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:00.543909 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:00.545628 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:00.545644 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:00.546375 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:00.546381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:00.546566 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:00.546612 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 12:47:00.560560 (+ 13948us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41985/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 12:47:00.560854 (+ 294us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.111.126:46771: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41985/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":314,"threads_started":1}
I20260811 12:47:00.562675 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 16639
I20260811 12:47:00.563813 16748 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:00.658021 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 16639
2026-08-11T12:47:00Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (732 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 12:47:00.719034 7613 mini_oidc.cc:152] Starting JWKS server
I20260811 12:47:00.721056 7613 webserver.cc:460] Webserver started at https://127.0.0.1:33599/ using document root <none> and password file <none>
I20260811 12:47:00.721354 7613 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 12:47:00.722059 7613 webserver.cc:460] Webserver started at http://127.0.0.1:44687/ using document root <none> and password file <none>
2026-08-11T12:47:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:47:00Z Disabled control of system clock
I20260811 12:47:00.757730 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:38741
--webserver_interface=127.7.111.126
--webserver_port=0
--builtin_ntp_servers=127.7.111.84:32933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:38741
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33599/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 12:47:01.084481 16819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:01.084789 16819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:01.084929 16819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:01.092041 16819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:47:01.092163 16819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:01.092232 16819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:01.092314 16819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:47:01.092381 16819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 12:47:01.092465 16819 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 12:47:01.092519 16819 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33599/jwks.json
I20260811 12:47:01.101635 16819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:32933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:38741
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:38741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33599/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:01.102991 16819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:01.104835 16819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:01.116537 16828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:01.116626 16827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:01.116634 16831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:01.117206 16819 server_base.cc:1034] running on GCE node
I20260811 12:47:01.118019 16819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:01.119372 16819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:01.120569 16819 hybrid_clock.cc:648] HybridClock initialized: now 1786452421120524 us; error 40 us; skew 500 ppm
I20260811 12:47:01.120919 16819 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:47:01.123157 16819 webserver.cc:460] Webserver started at http://127.7.111.126:46731/ using document root <none> and password file <none>
I20260811 12:47:01.123817 16819 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:01.123919 16819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:01.124244 16819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:01.127126 16819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "c61ebadbd2b24375819e52d5fdbdc6f1"
format_stamp: "Formatted at 2026-08-11 12:47:01 on dist-test-slave-f0tn"
server_key: "f907141d0ddb1269fa5ee8687d973eef"
server_key_iv: "7210cd410cf0259dc44e35c5a349cbfe"
server_key_version: "encryptionkey@0"
I20260811 12:47:01.127894 16819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "c61ebadbd2b24375819e52d5fdbdc6f1"
format_stamp: "Formatted at 2026-08-11 12:47:01 on dist-test-slave-f0tn"
server_key: "f907141d0ddb1269fa5ee8687d973eef"
server_key_iv: "7210cd410cf0259dc44e35c5a349cbfe"
server_key_version: "encryptionkey@0"
I20260811 12:47:01.132984 16819 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:47:01.136458 16841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:01.137845 16819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 12:47:01.138018 16819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "c61ebadbd2b24375819e52d5fdbdc6f1"
format_stamp: "Formatted at 2026-08-11 12:47:01 on dist-test-slave-f0tn"
server_key: "f907141d0ddb1269fa5ee8687d973eef"
server_key_iv: "7210cd410cf0259dc44e35c5a349cbfe"
server_key_version: "encryptionkey@0"
I20260811 12:47:01.138224 16819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:01.151917 16819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:01.152788 16819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:01.153095 16819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:01.170526 16819 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:38741
I20260811 12:47:01.170547 16921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:38741 every 8 connection(s)
I20260811 12:47:01.171939 16819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:47:01.175799 16923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:01.177148 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 16819
I20260811 12:47:01.177361 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:47:01.177995 7613 external_mini_cluster.cc:1428] Setting key d32d3e3727f13843d074c24257bd14c5
I20260811 12:47:01.187675 16923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c61ebadbd2b24375819e52d5fdbdc6f1: Bootstrap starting.
I20260811 12:47:01.191188 16923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c61ebadbd2b24375819e52d5fdbdc6f1: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:01.191550 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:01.179835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33718 (local address 127.7.111.126:38741)
0811 12:47:01.180853 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:01.180864 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:01.180918 (+ 54us) server_negotiation.cc:407] Connection header received
0811 12:47:01.182634 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:01.182659 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:01.183138 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:01.183458 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:01.184091 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:01.185472 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:01.187213 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:01.187896 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:01.188032 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:01.189217 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:01.189225 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:01.189241 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:47:01.189840 (+ 599us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:01.189940 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:47:01.189962 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:01.190047 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:01.190269 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 12:47:01.190568 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":347,"threads_started":1}
I20260811 12:47:01.192412 16923 log.cc:824] T 00000000000000000000000000000000 P c61ebadbd2b24375819e52d5fdbdc6f1: Log is configured to *not* fsync() on all Append() calls
I20260811 12:47:01.195528 16923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c61ebadbd2b24375819e52d5fdbdc6f1: No bootstrap required, opened a new log
I20260811 12:47:01.195523 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:47:01.202666 16923 raft_consensus.cc:348] T 00000000000000000000000000000000 P c61ebadbd2b24375819e52d5fdbdc6f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61ebadbd2b24375819e52d5fdbdc6f1" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38741 } }
I20260811 12:47:01.203011 16923 raft_consensus.cc:374] T 00000000000000000000000000000000 P c61ebadbd2b24375819e52d5fdbdc6f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:01.203109 16923 raft_consensus.cc:729] T 00000000000000000000000000000000 P c61ebadbd2b24375819e52d5fdbdc6f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c61ebadbd2b24375819e52d5fdbdc6f1, State: Initialized, Role: FOLLOWER
I20260811 12:47:01.203630 16923 consensus_queue.cc:260] T 00000000000000000000000000000000 P c61ebadbd2b24375819e52d5fdbdc6f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61ebadbd2b24375819e52d5fdbdc6f1" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38741 } }
I20260811 12:47:01.203853 16923 raft_consensus.cc:388] T 00000000000000000000000000000000 P c61ebadbd2b24375819e52d5fdbdc6f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:47:01.203980 16923 raft_consensus.cc:482] T 00000000000000000000000000000000 P c61ebadbd2b24375819e52d5fdbdc6f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:47:01.204129 16923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c61ebadbd2b24375819e52d5fdbdc6f1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:01.206492 16923 raft_consensus.cc:504] T 00000000000000000000000000000000 P c61ebadbd2b24375819e52d5fdbdc6f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61ebadbd2b24375819e52d5fdbdc6f1" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38741 } }
I20260811 12:47:01.206950 16923 leader_election.cc:302] T 00000000000000000000000000000000 P c61ebadbd2b24375819e52d5fdbdc6f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c61ebadbd2b24375819e52d5fdbdc6f1; no voters:
I20260811 12:47:01.207512 16923 leader_election.cc:288] T 00000000000000000000000000000000 P c61ebadbd2b24375819e52d5fdbdc6f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:47:01.207657 16935 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c61ebadbd2b24375819e52d5fdbdc6f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:47:01.208328 16935 raft_consensus.cc:686] T 00000000000000000000000000000000 P c61ebadbd2b24375819e52d5fdbdc6f1 [term 1 LEADER]: Becoming Leader. State: Replica: c61ebadbd2b24375819e52d5fdbdc6f1, State: Running, Role: LEADER
I20260811 12:47:01.208841 16935 consensus_queue.cc:237] T 00000000000000000000000000000000 P c61ebadbd2b24375819e52d5fdbdc6f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61ebadbd2b24375819e52d5fdbdc6f1" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38741 } }
I20260811 12:47:01.209537 16923 sys_catalog.cc:564] T 00000000000000000000000000000000 P c61ebadbd2b24375819e52d5fdbdc6f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:47:01.211575 16937 sys_catalog.cc:455] T 00000000000000000000000000000000 P c61ebadbd2b24375819e52d5fdbdc6f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c61ebadbd2b24375819e52d5fdbdc6f1. Latest consensus state: current_term: 1 leader_uuid: "c61ebadbd2b24375819e52d5fdbdc6f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61ebadbd2b24375819e52d5fdbdc6f1" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38741 } } }
I20260811 12:47:01.212103 16936 sys_catalog.cc:455] T 00000000000000000000000000000000 P c61ebadbd2b24375819e52d5fdbdc6f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c61ebadbd2b24375819e52d5fdbdc6f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61ebadbd2b24375819e52d5fdbdc6f1" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38741 } } }
I20260811 12:47:01.212409 16937 sys_catalog.cc:458] T 00000000000000000000000000000000 P c61ebadbd2b24375819e52d5fdbdc6f1 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:01.212415 16936 sys_catalog.cc:458] T 00000000000000000000000000000000 P c61ebadbd2b24375819e52d5fdbdc6f1 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:01.213353 16944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:47:01.219328 16944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:47:01.228307 16944 catalog_manager.cc:1347] Generated new cluster ID: cdd4ddb3a7314614a13514610b193105
I20260811 12:47:01.228426 16944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:47:01.238597 16944 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:47:01.239926 16944 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:47:01.246690 16944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c61ebadbd2b24375819e52d5fdbdc6f1: Generated new TSK 0
I20260811 12:47:01.247478 16944 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:47:01.273526 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:01.265081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33734 (local address 127.7.111.126:38741)
0811 12:47:01.265284 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:01.265290 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:01.265591 (+ 301us) server_negotiation.cc:407] Connection header received
0811 12:47:01.265851 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:01.265858 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:01.265956 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:01.266132 (+ 176us) server_negotiation.cc:226] Negotiated authn=JWT
0811 12:47:01.266544 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:01.267662 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:01.268989 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:01.269712 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:01.269844 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:01.270387 (+ 543us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 12:47:01.272838 (+ 2451us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33599/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 12:47:01.273169 (+ 331us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33734: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33599/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":79}
W20260811 12:47:01.273667 16930 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33734: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33599/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 12:47:01.275219 16972 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:01.264895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38741 (local address 127.0.0.1:33734)
0811 12:47:01.265491 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:01.265508 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:01.265632 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:01.266133 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:01.266136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:01.266145 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 12:47:01.266392 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:01.266400 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:01.267783 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:01.267786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:01.268843 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:01.268855 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:01.269899 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:01.269905 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:01.270042 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:01.270076 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 12:47:01.273047 (+ 2971us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33599/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 12:47:01.274785 (+ 1738us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.111.126:38741: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33599/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":1408,"thread_start_us":299,"threads_started":1}
I20260811 12:47:01.276623 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 16819
I20260811 12:47:01.277932 16914 generic_service.cc:196] Checking for leaks (request via RPC)
E20260811 12:47:01.284664 16974 webserver.cc:533] Webserver: sslize failed:
E20260811 12:47:01.284754 16974 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 12:47:01.369212 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 16819
2026-08-11T12:47:01Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (710 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T12:47:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:47:01Z Disabled control of system clock
I20260811 12:47:01.464190 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:40625
--webserver_interface=127.7.111.126
--webserver_port=0
--builtin_ntp_servers=127.7.111.84:42645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:40625
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786452375851340-7613-0/testchainca.cert with env {}
W20260811 12:47:01.786629 16980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:01.786935 16980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:01.787029 16980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:01.794044 16980 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:47:01.794142 16980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:01.794188 16980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:01.794242 16980 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:47:01.794286 16980 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 12:47:01.794345 16980 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 12:47:01.794384 16980 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 12:47:01.804100 16980 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:42645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:40625
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:40625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786452375851340-7613-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:01.805423 16980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:01.806700 16980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:01.818198 16986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:01.818243 16987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:01.818266 16989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:01.819545 16980 server_base.cc:1034] running on GCE node
I20260811 12:47:01.820250 16980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:01.821637 16980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:01.822845 16980 hybrid_clock.cc:648] HybridClock initialized: now 1786452421822791 us; error 44 us; skew 500 ppm
I20260811 12:47:01.823184 16980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:47:01.825323 16980 webserver.cc:460] Webserver started at http://127.7.111.126:39569/ using document root <none> and password file <none>
I20260811 12:47:01.826030 16980 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:01.826133 16980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:01.826443 16980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:01.829222 16980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "d0374ae1dba249ac9a8fb32f9d5579b9"
format_stamp: "Formatted at 2026-08-11 12:47:01 on dist-test-slave-f0tn"
server_key: "c50d4177691d870c1fe3b24848e0bc87"
server_key_iv: "0328ed08394143e8dd1028b021a6472c"
server_key_version: "encryptionkey@0"
I20260811 12:47:01.830039 16980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "d0374ae1dba249ac9a8fb32f9d5579b9"
format_stamp: "Formatted at 2026-08-11 12:47:01 on dist-test-slave-f0tn"
server_key: "c50d4177691d870c1fe3b24848e0bc87"
server_key_iv: "0328ed08394143e8dd1028b021a6472c"
server_key_version: "encryptionkey@0"
I20260811 12:47:01.835023 16980 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260811 12:47:01.838426 17001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:01.839738 16980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:47:01.839896 16980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "d0374ae1dba249ac9a8fb32f9d5579b9"
format_stamp: "Formatted at 2026-08-11 12:47:01 on dist-test-slave-f0tn"
server_key: "c50d4177691d870c1fe3b24848e0bc87"
server_key_iv: "0328ed08394143e8dd1028b021a6472c"
server_key_version: "encryptionkey@0"
I20260811 12:47:01.840095 16980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:01.852130 16980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:01.852926 16980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:01.853217 16980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:01.870546 16980 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:40625
I20260811 12:47:01.870589 17081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:40625 every 8 connection(s)
I20260811 12:47:01.871806 16980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:47:01.873816 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 16980
I20260811 12:47:01.873971 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:47:01.874650 7613 external_mini_cluster.cc:1428] Setting key ef276b5d4337ad2635c9986262ca96ad
I20260811 12:47:01.876171 17082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:01.888684 17082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0374ae1dba249ac9a8fb32f9d5579b9: Bootstrap starting.
I20260811 12:47:01.890705 17090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:01.876471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60978 (local address 127.7.111.126:40625)
0811 12:47:01.877465 (+ 994us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:01.877475 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:01.877518 (+ 43us) server_negotiation.cc:407] Connection header received
0811 12:47:01.879015 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:01.879044 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:01.879504 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:01.879831 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:01.880444 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:01.881827 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:01.883617 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:01.884264 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:01.884391 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:01.888435 (+ 4044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:01.888442 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:01.888455 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:47:01.889002 (+ 547us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:01.889074 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:47:01.889099 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:01.889197 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:01.889428 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 12:47:01.889720 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":336,"threads_started":1}
I20260811 12:47:01.892215 17082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0374ae1dba249ac9a8fb32f9d5579b9: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:01.893343 17082 log.cc:824] T 00000000000000000000000000000000 P d0374ae1dba249ac9a8fb32f9d5579b9: Log is configured to *not* fsync() on all Append() calls
I20260811 12:47:01.894546 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:47:01.896155 17082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0374ae1dba249ac9a8fb32f9d5579b9: No bootstrap required, opened a new log
I20260811 12:47:01.903791 17082 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0374ae1dba249ac9a8fb32f9d5579b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0374ae1dba249ac9a8fb32f9d5579b9" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40625 } }
I20260811 12:47:01.904124 17082 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0374ae1dba249ac9a8fb32f9d5579b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:01.904206 17082 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0374ae1dba249ac9a8fb32f9d5579b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0374ae1dba249ac9a8fb32f9d5579b9, State: Initialized, Role: FOLLOWER
I20260811 12:47:01.904671 17082 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0374ae1dba249ac9a8fb32f9d5579b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0374ae1dba249ac9a8fb32f9d5579b9" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40625 } }
I20260811 12:47:01.904873 17082 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0374ae1dba249ac9a8fb32f9d5579b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:47:01.904965 17082 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0374ae1dba249ac9a8fb32f9d5579b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:47:01.905094 17082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0374ae1dba249ac9a8fb32f9d5579b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:01.907397 17082 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0374ae1dba249ac9a8fb32f9d5579b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0374ae1dba249ac9a8fb32f9d5579b9" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40625 } }
I20260811 12:47:01.907832 17082 leader_election.cc:302] T 00000000000000000000000000000000 P d0374ae1dba249ac9a8fb32f9d5579b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0374ae1dba249ac9a8fb32f9d5579b9; no voters:
I20260811 12:47:01.908355 17082 leader_election.cc:288] T 00000000000000000000000000000000 P d0374ae1dba249ac9a8fb32f9d5579b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:47:01.908484 17092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0374ae1dba249ac9a8fb32f9d5579b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:47:01.909143 17092 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0374ae1dba249ac9a8fb32f9d5579b9 [term 1 LEADER]: Becoming Leader. State: Replica: d0374ae1dba249ac9a8fb32f9d5579b9, State: Running, Role: LEADER
I20260811 12:47:01.909653 17092 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0374ae1dba249ac9a8fb32f9d5579b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0374ae1dba249ac9a8fb32f9d5579b9" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40625 } }
I20260811 12:47:01.910245 17082 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0374ae1dba249ac9a8fb32f9d5579b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:47:01.912757 17093 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0374ae1dba249ac9a8fb32f9d5579b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0374ae1dba249ac9a8fb32f9d5579b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0374ae1dba249ac9a8fb32f9d5579b9" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40625 } } }
I20260811 12:47:01.913136 17093 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0374ae1dba249ac9a8fb32f9d5579b9 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:01.913439 17094 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0374ae1dba249ac9a8fb32f9d5579b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0374ae1dba249ac9a8fb32f9d5579b9. Latest consensus state: current_term: 1 leader_uuid: "d0374ae1dba249ac9a8fb32f9d5579b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0374ae1dba249ac9a8fb32f9d5579b9" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40625 } } }
I20260811 12:47:01.913755 17094 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0374ae1dba249ac9a8fb32f9d5579b9 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:01.915195 17107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:47:01.919787 17107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:47:01.927758 17107 catalog_manager.cc:1347] Generated new cluster ID: bdd3148209984d848b94817eeeaccdbc
I20260811 12:47:01.927861 17107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:47:01.957816 17107 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:47:01.959028 17107 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:47:01.967003 17107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0374ae1dba249ac9a8fb32f9d5579b9: Generated new TSK 0
I20260811 12:47:01.967800 17107 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:47:02.852643 17090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:01.989922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60990 (local address 127.7.111.126:40625)
0811 12:47:01.990112 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:01.990119 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:01.990571 (+ 452us) server_negotiation.cc:407] Connection header received
0811 12:47:01.990782 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:01.990792 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:01.990886 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:01.991057 (+ 171us) server_negotiation.cc:226] Negotiated authn=JWT
0811 12:47:01.991482 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:01.992600 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:01.994008 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:01.994700 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:01.994830 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:01.995382 (+ 552us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 12:47:02.851928 (+856546us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 12:47:02.852341 (+ 413us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60990: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 12:47:02.852813 17090 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60990: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 12:47:02.852967 17128 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:01.989840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40625 (local address 127.0.0.1:60990)
0811 12:47:01.990472 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:01.990487 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:01.990609 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:01.991052 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:01.991056 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:01.991064 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 12:47:01.991334 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:01.991342 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:01.992733 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:01.992736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:01.993865 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:01.993877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:01.994862 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:01.994867 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:01.995041 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:01.995083 (+ 42us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 12:47:02.852261 (+857178us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 12:47:02.852566 (+ 305us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.111.126:40625: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":343,"threads_started":1}
I20260811 12:47:02.854432 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 16980
I20260811 12:47:02.855490 17076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:02.958379 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 16980
2026-08-11T12:47:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1573 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (413 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (410 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:47:03 dist-test-slave-f0tn krb5kdc[17154](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:47:03 dist-test-slave-f0tn krb5kdc[17154](info): set up 2 sockets
Aug 11 12:47:03 dist-test-slave-f0tn krb5kdc[17154](info): commencing operation
krb5kdc: starting...
W20260811 12:47:05.021019 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:47:05 dist-test-slave-f0tn krb5kdc[17154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:47:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:47:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:47:05.246881 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:34137
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:45163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:34137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:47:05.570457 17170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:05.570752 17170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:05.570894 17170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:05.577893 17170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:47:05.578015 17170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:05.578094 17170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:05.578176 17170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:47:05.578238 17170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:47:05.587926 17170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:45163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:34137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:34137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:05.589284 17170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:05.590628 17170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:05.602187 17182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:05.602310 17183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:05.602264 17187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:05.604879 17170 server_base.cc:1034] running on GCE node
I20260811 12:47:05.605799 17170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:05.607314 17170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:05.608536 17170 hybrid_clock.cc:648] HybridClock initialized: now 1786452425608476 us; error 41 us; skew 500 ppm
I20260811 12:47:05.608886 17170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:05 dist-test-slave-f0tn krb5kdc[17154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452425, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:05.612288 17170 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:47:05.613811 17170 webserver.cc:460] Webserver started at http://127.7.111.126:35141/ using document root <none> and password file <none>
I20260811 12:47:05.614526 17170 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:05.614645 17170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:05.614957 17170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:05.618096 17170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "7561c465444c4d509c3e0be94c720c84"
format_stamp: "Formatted at 2026-08-11 12:47:05 on dist-test-slave-f0tn"
server_key: "45ee772aa6d0d5f1e548616af5109449"
server_key_iv: "51289eefe8c592495a25dfe9f9253219"
server_key_version: "encryptionkey@0"
I20260811 12:47:05.618897 17170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "7561c465444c4d509c3e0be94c720c84"
format_stamp: "Formatted at 2026-08-11 12:47:05 on dist-test-slave-f0tn"
server_key: "45ee772aa6d0d5f1e548616af5109449"
server_key_iv: "51289eefe8c592495a25dfe9f9253219"
server_key_version: "encryptionkey@0"
I20260811 12:47:05.624841 17170 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 12:47:05.628429 17199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:05.629824 17170 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:47:05.629981 17170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "7561c465444c4d509c3e0be94c720c84"
format_stamp: "Formatted at 2026-08-11 12:47:05 on dist-test-slave-f0tn"
server_key: "45ee772aa6d0d5f1e548616af5109449"
server_key_iv: "51289eefe8c592495a25dfe9f9253219"
server_key_version: "encryptionkey@0"
I20260811 12:47:05.630164 17170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:05.651117 17170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:05.654868 17170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:05.655128 17170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:05.671950 17170 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:34137
I20260811 12:47:05.671991 17273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:34137 every 8 connection(s)
I20260811 12:47:05.673300 17170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:47:05.676904 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 17170
I20260811 12:47:05.677121 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:47:05.677304 17277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:05.677989 7613 external_mini_cluster.cc:1428] Setting key 6fc45d008cfaffdbcf624b40df3abe63
I20260811 12:47:05.689841 17277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7561c465444c4d509c3e0be94c720c84: Bootstrap starting.
Aug 11 12:47:05 dist-test-slave-f0tn krb5kdc[17154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:47:05.693041 17277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7561c465444c4d509c3e0be94c720c84: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:05.694221 17277 log.cc:824] T 00000000000000000000000000000000 P 7561c465444c4d509c3e0be94c720c84: Log is configured to *not* fsync() on all Append() calls
I20260811 12:47:05.697324 17277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7561c465444c4d509c3e0be94c720c84: No bootstrap required, opened a new log
I20260811 12:47:05.699735 17282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:05.680110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60734 (local address 127.7.111.126:34137)
0811 12:47:05.680967 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:05.680980 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:05.681021 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:47:05.682629 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:05.682652 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:05.683038 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:05.683409 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:05.684857 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:05.686188 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:05.687435 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:05.688111 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:05.688290 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:05.692414 (+ 4124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:05.692422 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:05.692436 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:05.692471 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:05.695801 (+ 3330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:05.696586 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:05.696591 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:05.696597 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:05.696761 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:05.697152 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:05.697155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:05.697157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:05.697706 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:05.697921 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:05.698384 (+ 463us) server_negotiation.cc:299] Negotiation successful
0811 12:47:05.698704 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":383,"threads_started":1}
I20260811 12:47:05.704913 17277 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7561c465444c4d509c3e0be94c720c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7561c465444c4d509c3e0be94c720c84" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34137 } }
I20260811 12:47:05.705259 17277 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7561c465444c4d509c3e0be94c720c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:05.705351 17277 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7561c465444c4d509c3e0be94c720c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7561c465444c4d509c3e0be94c720c84, State: Initialized, Role: FOLLOWER
I20260811 12:47:05.705966 17277 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7561c465444c4d509c3e0be94c720c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7561c465444c4d509c3e0be94c720c84" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34137 } }
I20260811 12:47:05.706180 17277 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7561c465444c4d509c3e0be94c720c84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:47:05.706308 17277 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7561c465444c4d509c3e0be94c720c84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:47:05.706451 17277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7561c465444c4d509c3e0be94c720c84 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:05.708730 17277 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7561c465444c4d509c3e0be94c720c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7561c465444c4d509c3e0be94c720c84" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34137 } }
I20260811 12:47:05.709187 17277 leader_election.cc:302] T 00000000000000000000000000000000 P 7561c465444c4d509c3e0be94c720c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7561c465444c4d509c3e0be94c720c84; no voters:
I20260811 12:47:05.709757 17277 leader_election.cc:288] T 00000000000000000000000000000000 P 7561c465444c4d509c3e0be94c720c84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:47:05.709862 17284 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7561c465444c4d509c3e0be94c720c84 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:47:05.710502 17284 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7561c465444c4d509c3e0be94c720c84 [term 1 LEADER]: Becoming Leader. State: Replica: 7561c465444c4d509c3e0be94c720c84, State: Running, Role: LEADER
I20260811 12:47:05.711016 17284 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7561c465444c4d509c3e0be94c720c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7561c465444c4d509c3e0be94c720c84" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34137 } }
I20260811 12:47:05.711611 17277 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7561c465444c4d509c3e0be94c720c84 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:47:05.713935 17286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7561c465444c4d509c3e0be94c720c84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7561c465444c4d509c3e0be94c720c84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7561c465444c4d509c3e0be94c720c84" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34137 } } }
I20260811 12:47:05.713963 17287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7561c465444c4d509c3e0be94c720c84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7561c465444c4d509c3e0be94c720c84. Latest consensus state: current_term: 1 leader_uuid: "7561c465444c4d509c3e0be94c720c84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7561c465444c4d509c3e0be94c720c84" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34137 } } }
I20260811 12:47:05.714442 17286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7561c465444c4d509c3e0be94c720c84 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:05.714447 17287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7561c465444c4d509c3e0be94c720c84 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:05.715287 17301 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:47:05.719839 17301 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:47:05.727998 17301 catalog_manager.cc:1347] Generated new cluster ID: 831e82c63a2342758e677084c25587b4
I20260811 12:47:05.728091 17301 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:47:05.736773 17301 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:47:05.738137 17301 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:47:05.743557 17301 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7561c465444c4d509c3e0be94c720c84: Generated new TSK 0
I20260811 12:47:05.744479 17301 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.111.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
I20260811 12:47:05.810133 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34137
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:45163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:47:06.143150 17334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:06.143426 17334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:06.143560 17334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:06.150699 17334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:06.150820 17334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:06.150977 17334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:47:06.160943 17334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:45163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:06.162325 17334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:06.163796 17334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:06.176703 17341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:06.176750 17346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:06.176687 17343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:06.177901 17334 server_base.cc:1034] running on GCE node
I20260811 12:47:06.178474 17334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:06.179368 17334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:06.180563 17334 hybrid_clock.cc:648] HybridClock initialized: now 1786452426180515 us; error 36 us; skew 500 ppm
I20260811 12:47:06.180873 17334 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:06 dist-test-slave-f0tn krb5kdc[17154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452426, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:06.183956 17334 init.cc:377] Logged in from keytab as kudu/127.7.111.65@KRBTEST.COM (short username kudu)
I20260811 12:47:06.185329 17334 webserver.cc:460] Webserver started at http://127.7.111.65:43251/ using document root <none> and password file <none>
I20260811 12:47:06.186017 17334 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:06.186120 17334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:06.186386 17334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:06.189217 17334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "d99dea1a614348b89e965d65cfd7b6fd"
format_stamp: "Formatted at 2026-08-11 12:47:06 on dist-test-slave-f0tn"
server_key: "8360ccbb37bdbdacc4c18762b42e24c1"
server_key_iv: "f9d3c253e4ed0459067d048aa865c3f8"
server_key_version: "encryptionkey@0"
I20260811 12:47:06.189986 17334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "d99dea1a614348b89e965d65cfd7b6fd"
format_stamp: "Formatted at 2026-08-11 12:47:06 on dist-test-slave-f0tn"
server_key: "8360ccbb37bdbdacc4c18762b42e24c1"
server_key_iv: "f9d3c253e4ed0459067d048aa865c3f8"
server_key_version: "encryptionkey@0"
I20260811 12:47:06.194864 17334 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 12:47:06.198314 17353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:06.199618 17334 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:47:06.199754 17334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "d99dea1a614348b89e965d65cfd7b6fd"
format_stamp: "Formatted at 2026-08-11 12:47:06 on dist-test-slave-f0tn"
server_key: "8360ccbb37bdbdacc4c18762b42e24c1"
server_key_iv: "f9d3c253e4ed0459067d048aa865c3f8"
server_key_version: "encryptionkey@0"
I20260811 12:47:06.199915 17334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:06.222083 17334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:06.225955 17334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:06.226203 17334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:06.227118 17334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:06.228428 17334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:06.228506 17334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:06.228596 17334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:06.228638 17334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:06.259305 17334 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:41457
I20260811 12:47:06.259364 17555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:41457 every 8 connection(s)
I20260811 12:47:06.260604 17334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:47:06.265208 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 17334
I20260811 12:47:06.265429 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:47:06.266078 7613 external_mini_cluster.cc:1428] Setting key a94ae6911d979786eeebad489e040eeb
Aug 11 12:47:06 dist-test-slave-f0tn krb5kdc[17154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452426, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:47:06.280491 17559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:06.263241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:59805 (local address 127.7.111.126:34137)
0811 12:47:06.263705 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:06.263710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:06.264719 (+ 1009us) server_negotiation.cc:407] Connection header received
0811 12:47:06.266720 (+ 2001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:06.266724 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:06.266798 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:06.266921 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:06.268486 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:06.269223 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:06.270772 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:06.271184 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:06.271303 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:06.274992 (+ 3689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:06.274997 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:06.275000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:06.275033 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:06.277470 (+ 2437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:06.278189 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:06.278194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:06.278198 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:06.278346 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:06.278739 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:06.278744 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:06.278747 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:06.279100 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:06.279247 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:06.279916 (+ 669us) server_negotiation.cc:299] Negotiation successful
0811 12:47:06.280139 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":230,"threads_started":1}
I20260811 12:47:06.281255 17560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:06.263815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34137 (local address 127.7.111.65:59805)
0811 12:47:06.264562 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:06.264601 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:06.266422 (+ 1821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:06.267124 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:06.267133 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:06.267635 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:06.268315 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:06.268339 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:06.269368 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:06.269371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:06.270620 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:06.270633 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:06.271416 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:06.271424 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:06.271653 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:06.272291 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:06.272315 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:06.274748 (+ 2433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:06.277600 (+ 2852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:06.277609 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:06.277626 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:06.278052 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:06.278441 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:06.278444 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:06.278446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:06.278627 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:06.279218 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:06.279223 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:06.279532 (+ 309us) client_negotiation.cc:769] Sending connection context
0811 12:47:06.279753 (+ 221us) client_negotiation.cc:240] Negotiation successful
0811 12:47:06.280136 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":289,"threads_started":1}
I20260811 12:47:06.282793 17557 heartbeater.cc:347] Connected to a master server at 127.7.111.126:34137
I20260811 12:47:06.283079 17557 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:06.283793 17557 heartbeater.cc:511] Master 127.7.111.126:34137 requested a full tablet report, sending...
I20260811 12:47:06.285872 17229 ts_manager.cc:194] Registered new tserver with Master: d99dea1a614348b89e965d65cfd7b6fd (127.7.111.65:41457)
I20260811 12:47:06.287324 17229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:59805
WARNING: no policy specified for kudu/127.7.111.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
I20260811 12:47:06.322554 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34137
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:45163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:47:06.655174 17563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:06.655479 17563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:06.655606 17563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:06.663022 17563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:06.663156 17563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:06.663302 17563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:47:06.673483 17563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:45163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:06.674856 17563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:06.676312 17563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:06.689666 17571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:06.689704 17574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:06.690237 17578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:06.691025 17563 server_base.cc:1034] running on GCE node
I20260811 12:47:06.691673 17563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:06.692636 17563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:06.693843 17563 hybrid_clock.cc:648] HybridClock initialized: now 1786452426693798 us; error 35 us; skew 500 ppm
I20260811 12:47:06.694162 17563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:06 dist-test-slave-f0tn krb5kdc[17154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452426, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:06.697322 17563 init.cc:377] Logged in from keytab as kudu/127.7.111.66@KRBTEST.COM (short username kudu)
I20260811 12:47:06.698894 17563 webserver.cc:460] Webserver started at http://127.7.111.66:33549/ using document root <none> and password file <none>
I20260811 12:47:06.699604 17563 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:06.699729 17563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:06.700039 17563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:06.703209 17563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "dde9d43dde694e7198ee9d2a5fa39e20"
format_stamp: "Formatted at 2026-08-11 12:47:06 on dist-test-slave-f0tn"
server_key: "99498567da2031f43f94e7542fa334b0"
server_key_iv: "c3001ef4b333be41954858b8965c43c1"
server_key_version: "encryptionkey@0"
I20260811 12:47:06.703979 17563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "dde9d43dde694e7198ee9d2a5fa39e20"
format_stamp: "Formatted at 2026-08-11 12:47:06 on dist-test-slave-f0tn"
server_key: "99498567da2031f43f94e7542fa334b0"
server_key_iv: "c3001ef4b333be41954858b8965c43c1"
server_key_version: "encryptionkey@0"
I20260811 12:47:06.709120 17563 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:47:06.712652 17593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:06.714062 17563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:47:06.714234 17563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "dde9d43dde694e7198ee9d2a5fa39e20"
format_stamp: "Formatted at 2026-08-11 12:47:06 on dist-test-slave-f0tn"
server_key: "99498567da2031f43f94e7542fa334b0"
server_key_iv: "c3001ef4b333be41954858b8965c43c1"
server_key_version: "encryptionkey@0"
I20260811 12:47:06.714421 17563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:06.745591 17563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:06.749565 17563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:06.749846 17563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:06.750833 17563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:06.752183 17563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:06.752262 17563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:06.752394 17563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:06.752449 17563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:06.783099 17563 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:39021
I20260811 12:47:06.783154 17771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:39021 every 8 connection(s)
I20260811 12:47:06.784387 17563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:47:06.787667 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 17563
I20260811 12:47:06.787834 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:47:06.788291 7613 external_mini_cluster.cc:1428] Setting key b363af4df00a1bde15becd7e05891e9a
Aug 11 12:47:06 dist-test-slave-f0tn krb5kdc[17154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452426, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:47:06.804597 17776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:06.786874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:47433 (local address 127.7.111.126:34137)
0811 12:47:06.787295 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:06.787300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:06.790364 (+ 3064us) server_negotiation.cc:407] Connection header received
0811 12:47:06.790442 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:06.790447 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:06.790529 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:06.790646 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:06.792063 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:06.793066 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:06.794620 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:06.795190 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:06.795310 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:06.798567 (+ 3257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:06.798574 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:06.798578 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:06.798615 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:06.801168 (+ 2553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:06.802093 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:06.802097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:06.802099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:06.802249 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:06.802708 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:06.802713 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:06.802716 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:06.803042 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:06.803151 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:06.804107 (+ 956us) server_negotiation.cc:299] Negotiation successful
0811 12:47:06.804294 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":212,"threads_started":1}
I20260811 12:47:06.805979 17777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:06.787447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34137 (local address 127.7.111.66:47433)
0811 12:47:06.788238 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:06.788284 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:06.790229 (+ 1945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:06.790824 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:06.790831 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:06.791333 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:06.791888 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:06.791905 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:06.793207 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:06.793211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:06.794479 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:06.794492 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:06.795374 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:06.795379 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:06.795564 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:06.796173 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:06.796197 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:06.798348 (+ 2151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:06.801325 (+ 2977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:06.801335 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:06.801353 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:06.801984 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:06.802355 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:06.802359 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:06.802362 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:06.802602 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:06.803157 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:06.803165 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:06.803596 (+ 431us) client_negotiation.cc:769] Sending connection context
0811 12:47:06.803912 (+ 316us) client_negotiation.cc:240] Negotiation successful
0811 12:47:06.804521 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":330,"threads_started":1}
I20260811 12:47:06.807523 17773 heartbeater.cc:347] Connected to a master server at 127.7.111.126:34137
I20260811 12:47:06.807827 17773 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:06.808588 17773 heartbeater.cc:511] Master 127.7.111.126:34137 requested a full tablet report, sending...
I20260811 12:47:06.810052 17229 ts_manager.cc:194] Registered new tserver with Master: dde9d43dde694e7198ee9d2a5fa39e20 (127.7.111.66:39021)
I20260811 12:47:06.810864 17229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:47433
WARNING: no policy specified for kudu/127.7.111.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
I20260811 12:47:06.846884 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34137
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:45163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:47:07.173509 17780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:07.173777 17780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:07.173915 17780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:07.180950 17780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:07.181061 17780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:07.181214 17780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:47:07.190683 17780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:45163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:07.191900 17780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:07.193186 17780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:07.205688 17789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:07.206288 17791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:07.206710 17793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:07.206849 17780 server_base.cc:1034] running on GCE node
I20260811 12:47:07.207491 17780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:07.208468 17780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:07.209659 17780 hybrid_clock.cc:648] HybridClock initialized: now 1786452427209597 us; error 52 us; skew 500 ppm
I20260811 12:47:07.209971 17780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:07 dist-test-slave-f0tn krb5kdc[17154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452427, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:07.216890 17780 init.cc:377] Logged in from keytab as kudu/127.7.111.67@KRBTEST.COM (short username kudu)
I20260811 12:47:07.218587 17780 webserver.cc:460] Webserver started at http://127.7.111.67:37051/ using document root <none> and password file <none>
I20260811 12:47:07.219416 17780 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:07.219564 17780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:07.219899 17780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:07.223001 17780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "e079ec301faf459391f0e8d85b74702e"
format_stamp: "Formatted at 2026-08-11 12:47:07 on dist-test-slave-f0tn"
server_key: "dbc6fb34641210a95e6b92cc634f14af"
server_key_iv: "a116f5fc042bca2c31ff3613e87d3fbb"
server_key_version: "encryptionkey@0"
I20260811 12:47:07.223774 17780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "e079ec301faf459391f0e8d85b74702e"
format_stamp: "Formatted at 2026-08-11 12:47:07 on dist-test-slave-f0tn"
server_key: "dbc6fb34641210a95e6b92cc634f14af"
server_key_iv: "a116f5fc042bca2c31ff3613e87d3fbb"
server_key_version: "encryptionkey@0"
I20260811 12:47:07.228971 17780 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:47:07.232494 17802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:07.233953 17780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:47:07.234094 17780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "e079ec301faf459391f0e8d85b74702e"
format_stamp: "Formatted at 2026-08-11 12:47:07 on dist-test-slave-f0tn"
server_key: "dbc6fb34641210a95e6b92cc634f14af"
server_key_iv: "a116f5fc042bca2c31ff3613e87d3fbb"
server_key_version: "encryptionkey@0"
I20260811 12:47:07.234270 17780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:07.250612 17780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:07.254410 17780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:07.254691 17780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:07.255627 17780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:07.256925 17780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:07.257001 17780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:07.257095 17780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:07.257138 17780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:07.287654 17780 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:41865
I20260811 12:47:07.287693 17975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:41865 every 8 connection(s)
I20260811 12:47:07.288920 17780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:47:07.289779 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 17780
I20260811 12:47:07.289990 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:47:07.290632 7613 external_mini_cluster.cc:1428] Setting key f1ecd11e4e383a837441b8e649653e85
I20260811 12:47:07.290700 17557 heartbeater.cc:503] Master 127.7.111.126:34137 was elected leader, sending a full tablet report...
Aug 11 12:47:07 dist-test-slave-f0tn krb5kdc[17154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452427, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:47:07.309626 17776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:07.291812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:45051 (local address 127.7.111.126:34137)
0811 12:47:07.291992 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:07.291998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:07.293763 (+ 1765us) server_negotiation.cc:407] Connection header received
0811 12:47:07.295708 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:07.295712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:07.295779 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:07.295850 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:07.297422 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:07.298184 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:07.299755 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:07.300142 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:07.300231 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:07.303610 (+ 3379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:07.303614 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:07.303617 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:07.303649 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:07.306909 (+ 3260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:07.307601 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:07.307606 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:07.307608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:07.307733 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:07.308103 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:07.308107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:07.308109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:07.308404 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:07.308522 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:07.309154 (+ 632us) server_negotiation.cc:299] Negotiation successful
0811 12:47:07.309322 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:47:07.310582 17981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:07.292617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34137 (local address 127.7.111.67:45051)
0811 12:47:07.293620 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:07.293670 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:07.295429 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:07.296065 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:07.296073 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:07.296615 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:07.297234 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:07.297254 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:07.298348 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:07.298352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:07.299630 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:07.299643 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:07.300312 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:07.300317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:07.300517 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:07.301145 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:07.301172 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:07.303410 (+ 2238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:07.307041 (+ 3631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:07.307048 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:07.307060 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:07.307473 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:07.307825 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:07.307828 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:07.307830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:07.308006 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:07.308500 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:07.308506 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:07.308824 (+ 318us) client_negotiation.cc:769] Sending connection context
0811 12:47:07.309053 (+ 229us) client_negotiation.cc:240] Negotiation successful
0811 12:47:07.309485 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":393,"threads_started":1}
I20260811 12:47:07.311887 17977 heartbeater.cc:347] Connected to a master server at 127.7.111.126:34137
I20260811 12:47:07.312179 17977 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:07.313221 17977 heartbeater.cc:511] Master 127.7.111.126:34137 requested a full tablet report, sending...
I20260811 12:47:07.314633 17229 ts_manager.cc:194] Registered new tserver with Master: e079ec301faf459391f0e8d85b74702e (127.7.111.67:41865)
I20260811 12:47:07.315430 17229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:45051
I20260811 12:47:07.318517 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/test-admin.keytab.
Aug 11 12:47:07 dist-test-slave-f0tn krb5kdc[17154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:07.350538 7613 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 12:47:07.355954 17990 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:07.353309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41388 (local address 127.7.111.65:41457)
0811 12:47:07.354004 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:07.354011 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:07.354037 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:47:07.354104 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:07.354110 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:07.354324 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:07.354493 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:07.355546 (+ 1053us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41388: BlockingRecv error: recv got EOF from 127.0.0.1:41388 (error 108)
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":328,"threads_started":1}
W20260811 12:47:07.356037 7613 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/krb5cc' of type FILE
Aug 11 12:47:07 dist-test-slave-f0tn krb5kdc[17154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:07.357620 7613 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 12:47:07 dist-test-slave-f0tn krb5kdc[17154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:07.358904 7613 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 12:47:07 dist-test-slave-f0tn krb5kdc[17154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.65@KRBTEST.COM
I20260811 12:47:07.373214 17990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:07.360508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41402 (local address 127.7.111.65:41457)
0811 12:47:07.360688 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:07.360693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:07.360850 (+ 157us) server_negotiation.cc:407] Connection header received
0811 12:47:07.360975 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:07.360979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:07.361035 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:07.361111 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:07.362116 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:07.363071 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:07.364243 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:07.364784 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:07.364877 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:07.368099 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:07.368110 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:07.368141 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:07.368178 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:07.370840 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:07.371432 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:07.371438 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:07.371441 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:07.371559 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:07.371874 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:07.371878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:07.371881 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:07.372249 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:07.372472 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:07.372690 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 12:47:07.372917 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 12:47:07.378821 17990 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:07.377075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41416 (local address 127.7.111.65:41457)
0811 12:47:07.377259 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:07.377264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:07.377603 (+ 339us) server_negotiation.cc:407] Connection header received
0811 12:47:07.377769 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:07.377773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:07.377829 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:07.377904 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:07.378614 (+ 710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41416: BlockingRecv error: recv got EOF from 127.0.0.1:41416 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 12:47:07.379129 7613 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/krb5cc' of type FILE
Aug 11 12:47:07 dist-test-slave-f0tn krb5kdc[17154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:07.380548 7613 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 12:47:07 dist-test-slave-f0tn krb5kdc[17154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:07.381814 7613 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 12:47:07 dist-test-slave-f0tn krb5kdc[17154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.65@KRBTEST.COM
I20260811 12:47:07.394160 17990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:07.383034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41424 (local address 127.7.111.65:41457)
0811 12:47:07.383209 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:07.383214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:07.383351 (+ 137us) server_negotiation.cc:407] Connection header received
0811 12:47:07.383454 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:07.383457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:07.383507 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:07.383578 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:07.384408 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:07.385087 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:07.386238 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:07.386618 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:07.386731 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:07.389860 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:07.389865 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:07.389868 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:07.389898 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:07.392108 (+ 2210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:07.392643 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:07.392647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:07.392649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:07.392773 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:07.393053 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:07.393056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:07.393058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:07.393288 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:07.393516 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:07.393731 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 12:47:07.393890 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 12:47:07.398312 17990 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:07.396816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41438 (local address 127.7.111.65:41457)
0811 12:47:07.397009 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:07.397014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:07.397217 (+ 203us) server_negotiation.cc:407] Connection header received
0811 12:47:07.397323 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:07.397326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:07.397446 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:07.397518 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:07.398100 (+ 582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41438: BlockingRecv error: recv got EOF from 127.0.0.1:41438 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 12:47:07.398597 7613 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786452375851340-7613-0/krb5kdc/krb5cc' of type FILE
Aug 11 12:47:07 dist-test-slave-f0tn krb5kdc[17154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:07.399946 7613 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 12:47:07 dist-test-slave-f0tn krb5kdc[17154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:07.401207 7613 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 12:47:07 dist-test-slave-f0tn krb5kdc[17154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.65@KRBTEST.COM
I20260811 12:47:07.414674 17990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:07.402415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41442 (local address 127.7.111.65:41457)
0811 12:47:07.402603 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:07.402608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:07.402839 (+ 231us) server_negotiation.cc:407] Connection header received
0811 12:47:07.402965 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:07.402969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:07.403022 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:07.403090 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:07.403998 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:07.404761 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:07.406047 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:07.406457 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:07.406545 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:07.409708 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:07.409712 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:07.409715 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:07.409746 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:07.412127 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:07.412808 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:07.412812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:07.412815 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:07.412965 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:07.413317 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:07.413321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:07.413323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:07.413617 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:07.413913 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:07.414181 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 12:47:07.414390 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:47:07.416216 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 17334
I20260811 12:47:07.427861 17990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:07.416852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41448 (local address 127.7.111.65:41457)
0811 12:47:07.417037 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:07.417044 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:07.417315 (+ 271us) server_negotiation.cc:407] Connection header received
0811 12:47:07.417466 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:07.417469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:07.417524 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:07.417594 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:07.418460 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:07.419281 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:07.420495 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:07.420962 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:07.421070 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:07.423358 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:07.423362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:07.423366 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:07.423399 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:07.425712 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:07.426273 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:07.426277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:07.426279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:07.426410 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:07.426678 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:07.426681 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:07.426683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:07.426919 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:07.427114 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:07.427364 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 12:47:07.427531 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:47:07.428465 17550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:07.510509 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 17334
I20260811 12:47:07.535300 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 17563
Aug 11 12:47:07 dist-test-slave-f0tn krb5kdc[17154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.66@KRBTEST.COM
I20260811 12:47:07.550258 18008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:07.536201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53250 (local address 127.7.111.66:39021)
0811 12:47:07.536740 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:07.536747 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:07.536774 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:47:07.536844 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:07.536851 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:07.537056 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:07.537192 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:07.538176 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:07.539271 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:07.540437 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:07.540964 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:07.541057 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:07.544780 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:07.544792 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:07.544807 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:07.544839 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:07.547531 (+ 2692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:07.548187 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:07.548195 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:07.548204 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:07.548328 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:07.548706 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:07.548710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:07.548712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:07.549043 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:07.549330 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:07.549674 (+ 344us) server_negotiation.cc:299] Negotiation successful
0811 12:47:07.549926 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":249,"threads_started":1}
I20260811 12:47:07.551524 17765 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:07.634472 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 17563
I20260811 12:47:07.659257 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 17780
Aug 11 12:47:07 dist-test-slave-f0tn krb5kdc[17154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.67@KRBTEST.COM
I20260811 12:47:07.675209 18015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:07.660242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51840 (local address 127.7.111.67:41865)
0811 12:47:07.660935 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:07.660942 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:07.660972 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:47:07.661042 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:07.661050 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:07.661259 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:07.661438 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:07.662563 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:07.663585 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:07.664878 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:07.665491 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:07.665610 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:07.669443 (+ 3833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:07.669458 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:07.669476 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:07.669511 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:07.672331 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:07.672959 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:07.672968 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:07.672972 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:07.673119 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:07.673610 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:07.673615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:07.673619 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:07.674045 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:07.674344 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:07.674540 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 12:47:07.674816 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":367,"threads_started":1}
I20260811 12:47:07.676616 17969 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:07.758639 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 17780
I20260811 12:47:07.781032 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 17170
I20260811 12:47:07.782238 17268 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:07.885861 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 17170
2026-08-11T12:47:07Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4111 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:47:08 dist-test-slave-f0tn krb5kdc[18024](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:47:08 dist-test-slave-f0tn krb5kdc[18024](info): set up 2 sockets
Aug 11 12:47:08 dist-test-slave-f0tn krb5kdc[18024](info): commencing operation
krb5kdc: starting...
W20260811 12:47:09.726521 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.787s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:47:09 dist-test-slave-f0tn krb5kdc[18024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:47:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:47:09Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.126.keytab.
Entry for principal oryx/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.126.keytab.
I20260811 12:47:09.970443 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:34613
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.126.keytab
--principal=oryx/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:42611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:34613
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:47:10.296914 18041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:10.297194 18041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:10.297297 18041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:10.304304 18041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:47:10.304407 18041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:10.304452 18041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:10.304508 18041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:47:10.304551 18041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:47:10.314275 18041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:42611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:34613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.126.keytab
--principal=oryx/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:34613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:10.315549 18041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:10.316828 18041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:10.328109 18047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:10.328284 18046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:10.328158 18051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:10.329142 18041 server_base.cc:1034] running on GCE node
I20260811 12:47:10.329960 18041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:10.331404 18041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:10.332612 18041 hybrid_clock.cc:648] HybridClock initialized: now 1786452430332558 us; error 44 us; skew 500 ppm
I20260811 12:47:10.332962 18041 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:10 dist-test-slave-f0tn krb5kdc[18024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452430, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:10.336079 18041 init.cc:377] Logged in from keytab as oryx/127.7.111.126@KRBTEST.COM (short username oryx)
I20260811 12:47:10.337558 18041 webserver.cc:460] Webserver started at http://127.7.111.126:37581/ using document root <none> and password file <none>
I20260811 12:47:10.338251 18041 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:10.338399 18041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:10.338745 18041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:10.341704 18041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "8acd3cadf1424c2792ed3fc7e4c5f506"
format_stamp: "Formatted at 2026-08-11 12:47:10 on dist-test-slave-f0tn"
server_key: "2b5f8d113e0a28d079cf77529df57c0a"
server_key_iv: "2646fa347b523dcbd0de10aa21a670a1"
server_key_version: "encryptionkey@0"
I20260811 12:47:10.342430 18041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "8acd3cadf1424c2792ed3fc7e4c5f506"
format_stamp: "Formatted at 2026-08-11 12:47:10 on dist-test-slave-f0tn"
server_key: "2b5f8d113e0a28d079cf77529df57c0a"
server_key_iv: "2646fa347b523dcbd0de10aa21a670a1"
server_key_version: "encryptionkey@0"
I20260811 12:47:10.347365 18041 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:47:10.350627 18061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:10.351905 18041 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:47:10.352064 18041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "8acd3cadf1424c2792ed3fc7e4c5f506"
format_stamp: "Formatted at 2026-08-11 12:47:10 on dist-test-slave-f0tn"
server_key: "2b5f8d113e0a28d079cf77529df57c0a"
server_key_iv: "2646fa347b523dcbd0de10aa21a670a1"
server_key_version: "encryptionkey@0"
I20260811 12:47:10.352240 18041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:10.373560 18041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:10.377686 18041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:10.377947 18041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:10.395391 18041 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:34613
I20260811 12:47:10.395411 18141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:34613 every 8 connection(s)
I20260811 12:47:10.396723 18041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:47:10.400346 18142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:10.402727 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 18041
I20260811 12:47:10.402930 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:47:10.403779 7613 external_mini_cluster.cc:1428] Setting key 0175a73b142002fa53e55d78b7df5620
I20260811 12:47:10.412597 18142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506: Bootstrap starting.
I20260811 12:47:10.416265 18142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:10.417564 18142 log.cc:824] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506: Log is configured to *not* fsync() on all Append() calls
Aug 11 12:47:10 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM
I20260811 12:47:10.420542 18142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506: No bootstrap required, opened a new log
I20260811 12:47:10.426481 18145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:10.405942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42534 (local address 127.7.111.126:34613)
0811 12:47:10.406701 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:10.406712 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:10.406756 (+ 44us) server_negotiation.cc:407] Connection header received
0811 12:47:10.408415 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:10.408438 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:10.408818 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:10.409136 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:10.410955 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:10.412238 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:10.413909 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:10.414584 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:10.414730 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:10.419516 (+ 4786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:10.419524 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:10.419539 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:10.419579 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:10.422603 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:10.423242 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:10.423248 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:10.423254 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:10.423422 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:10.423808 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:10.423812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:10.423814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:10.424324 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:10.424666 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:10.425066 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 12:47:10.425507 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":348,"threads_started":1}
I20260811 12:47:10.428244 18142 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } }
I20260811 12:47:10.428606 18142 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:10.428712 18142 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8acd3cadf1424c2792ed3fc7e4c5f506, State: Initialized, Role: FOLLOWER
I20260811 12:47:10.429311 18142 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } }
I20260811 12:47:10.429563 18142 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:47:10.429689 18142 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:47:10.429842 18142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:10.431962 18142 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } }
I20260811 12:47:10.432449 18142 leader_election.cc:302] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8acd3cadf1424c2792ed3fc7e4c5f506; no voters:
I20260811 12:47:10.433022 18142 leader_election.cc:288] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:47:10.433146 18147 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:47:10.433827 18147 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [term 1 LEADER]: Becoming Leader. State: Replica: 8acd3cadf1424c2792ed3fc7e4c5f506, State: Running, Role: LEADER
I20260811 12:47:10.434417 18147 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } }
I20260811 12:47:10.434988 18142 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:47:10.437103 18149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8acd3cadf1424c2792ed3fc7e4c5f506. Latest consensus state: current_term: 1 leader_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } }
I20260811 12:47:10.437479 18149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:10.437970 18148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } }
I20260811 12:47:10.438279 18148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:10.442435 18154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:47:10.447710 18154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:47:10.455304 18154 catalog_manager.cc:1347] Generated new cluster ID: 53e0569ae84d4f13b3a2305a0117f25e
I20260811 12:47:10.455407 18154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:47:10.493278 18154 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:47:10.495149 18154 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:47:10.518466 18154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506: Generated new TSK 0
I20260811 12:47:10.519536 18154 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.111.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.111.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.111.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.65.keytab.
Entry for principal oryx/127.7.111.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.65.keytab.
I20260811 12:47:10.591301 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34613
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.65.keytab
--principal=oryx/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:42611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:47:10.931970 18177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:10.932277 18177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:10.932425 18177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:10.940412 18177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:10.940546 18177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:10.940713 18177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:47:10.950971 18177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:42611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.65.keytab
--principal=oryx/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:10.952309 18177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:10.953760 18177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:10.967495 18189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:10.967577 18192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:10.967571 18187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:10.968113 18177 server_base.cc:1034] running on GCE node
I20260811 12:47:10.968727 18177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:10.969647 18177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:10.970853 18177 hybrid_clock.cc:648] HybridClock initialized: now 1786452430970798 us; error 39 us; skew 500 ppm
I20260811 12:47:10.971145 18177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:10 dist-test-slave-f0tn krb5kdc[18024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452430, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:10.974293 18177 init.cc:377] Logged in from keytab as oryx/127.7.111.65@KRBTEST.COM (short username oryx)
I20260811 12:47:10.975754 18177 webserver.cc:460] Webserver started at http://127.7.111.65:43627/ using document root <none> and password file <none>
I20260811 12:47:10.976459 18177 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:10.976576 18177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:10.976871 18177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:10.979820 18177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "783323bb150f42aeb023d8577443ffc0"
format_stamp: "Formatted at 2026-08-11 12:47:10 on dist-test-slave-f0tn"
server_key: "791dc13123f155ec0d70b67febe5d42d"
server_key_iv: "0bf3f2d4eea2abec044068748a6bc8a6"
server_key_version: "encryptionkey@0"
I20260811 12:47:10.980547 18177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "783323bb150f42aeb023d8577443ffc0"
format_stamp: "Formatted at 2026-08-11 12:47:10 on dist-test-slave-f0tn"
server_key: "791dc13123f155ec0d70b67febe5d42d"
server_key_iv: "0bf3f2d4eea2abec044068748a6bc8a6"
server_key_version: "encryptionkey@0"
I20260811 12:47:10.985903 18177 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:47:10.989638 18202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:10.991076 18177 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:47:10.991250 18177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "783323bb150f42aeb023d8577443ffc0"
format_stamp: "Formatted at 2026-08-11 12:47:10 on dist-test-slave-f0tn"
server_key: "791dc13123f155ec0d70b67febe5d42d"
server_key_iv: "0bf3f2d4eea2abec044068748a6bc8a6"
server_key_version: "encryptionkey@0"
I20260811 12:47:10.991459 18177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:11.005745 18177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:11.010437 18177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:11.010708 18177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:11.011683 18177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:11.013083 18177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:11.013157 18177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:11.013252 18177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:11.013296 18177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:11.043502 18177 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:33335
I20260811 12:47:11.043555 18379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:33335 every 8 connection(s)
I20260811 12:47:11.044744 18177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:47:11.047995 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 18177
I20260811 12:47:11.048236 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:47:11.049082 7613 external_mini_cluster.cc:1428] Setting key 5337eb1b09db7fc6275a9c55c1cffe07
Aug 11 12:47:11 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452430, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM
I20260811 12:47:11.066250 18383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:11.047224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:46043 (local address 127.7.111.126:34613)
0811 12:47:11.047650 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:11.047655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:11.048759 (+ 1104us) server_negotiation.cc:407] Connection header received
0811 12:47:11.050804 (+ 2045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:11.050810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:11.050895 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:11.051007 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:11.052997 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:11.053754 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:11.055819 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:11.056258 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:11.056353 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:11.060324 (+ 3971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:11.060329 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:11.060332 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:11.060365 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:11.062945 (+ 2580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:11.063805 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:11.063809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:11.063812 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:11.063951 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:11.064417 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:11.064420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:11.064423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:11.064712 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:11.064828 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:11.065778 (+ 950us) server_negotiation.cc:299] Negotiation successful
0811 12:47:11.065957 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":218,"threads_started":1}
I20260811 12:47:11.067660 18385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:11.047789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.65:46043)
0811 12:47:11.048602 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:11.048641 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:11.050534 (+ 1893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:11.051277 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:11.051291 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:11.052048 (+ 757us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:11.052814 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:11.052839 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:11.053908 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:11.053913 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:11.055659 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:11.055698 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:11.056475 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:11.056481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:11.056749 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:11.057482 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:11.057509 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:11.060106 (+ 2597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:11.063087 (+ 2981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:11.063097 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:11.063114 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:11.063667 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:11.064057 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:11.064062 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:11.064066 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:11.064316 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:11.064838 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:11.064846 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:11.065345 (+ 499us) client_negotiation.cc:769] Sending connection context
0811 12:47:11.065706 (+ 361us) client_negotiation.cc:240] Negotiation successful
0811 12:47:11.066236 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":333,"threads_started":1}
I20260811 12:47:11.069221 18381 heartbeater.cc:347] Connected to a master server at 127.7.111.126:34613
I20260811 12:47:11.069568 18381 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:11.070323 18381 heartbeater.cc:511] Master 127.7.111.126:34613 requested a full tablet report, sending...
I20260811 12:47:11.072398 18089 ts_manager.cc:194] Registered new tserver with Master: 783323bb150f42aeb023d8577443ffc0 (127.7.111.65:33335)
I20260811 12:47:11.073930 18089 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:46043
WARNING: no policy specified for oryx/127.7.111.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.111.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.111.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.66.keytab.
Entry for principal oryx/127.7.111.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.66.keytab.
I20260811 12:47:11.106879 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34613
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.66.keytab
--principal=oryx/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:42611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:47:11.438828 18388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:11.439090 18388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:11.439206 18388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:11.446324 18388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:11.446420 18388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:11.446537 18388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:47:11.455819 18388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:42611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.66.keytab
--principal=oryx/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:11.457000 18388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:11.458292 18388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:11.471427 18397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:11.471467 18399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:11.471388 18404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:11.471947 18388 server_base.cc:1034] running on GCE node
I20260811 12:47:11.472584 18388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:11.473608 18388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:11.474822 18388 hybrid_clock.cc:648] HybridClock initialized: now 1786452431474766 us; error 50 us; skew 500 ppm
I20260811 12:47:11.475127 18388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:11 dist-test-slave-f0tn krb5kdc[18024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452431, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:11.478121 18388 init.cc:377] Logged in from keytab as oryx/127.7.111.66@KRBTEST.COM (short username oryx)
I20260811 12:47:11.479512 18388 webserver.cc:460] Webserver started at http://127.7.111.66:41233/ using document root <none> and password file <none>
I20260811 12:47:11.480211 18388 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:11.480327 18388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:11.480618 18388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:11.483572 18388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "16939728527c4f3287b355862cc6a6f0"
format_stamp: "Formatted at 2026-08-11 12:47:11 on dist-test-slave-f0tn"
server_key: "db0873f4f8df918fb90fb9a2fd55ab54"
server_key_iv: "1ec8adb18ee46e42bfaba198edcb27f0"
server_key_version: "encryptionkey@0"
I20260811 12:47:11.484314 18388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "16939728527c4f3287b355862cc6a6f0"
format_stamp: "Formatted at 2026-08-11 12:47:11 on dist-test-slave-f0tn"
server_key: "db0873f4f8df918fb90fb9a2fd55ab54"
server_key_iv: "1ec8adb18ee46e42bfaba198edcb27f0"
server_key_version: "encryptionkey@0"
I20260811 12:47:11.489439 18388 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:47:11.492720 18416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:11.494060 18388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 12:47:11.494213 18388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "16939728527c4f3287b355862cc6a6f0"
format_stamp: "Formatted at 2026-08-11 12:47:11 on dist-test-slave-f0tn"
server_key: "db0873f4f8df918fb90fb9a2fd55ab54"
server_key_iv: "1ec8adb18ee46e42bfaba198edcb27f0"
server_key_version: "encryptionkey@0"
I20260811 12:47:11.494400 18388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:11.507300 18388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:11.510964 18388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:11.511231 18388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:11.512145 18388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:11.513471 18388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:11.513551 18388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:11.513680 18388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:11.513741 18388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:11.544167 18388 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:46081
I20260811 12:47:11.544203 18579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:46081 every 8 connection(s)
I20260811 12:47:11.545506 18388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:47:11.551363 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 18388
I20260811 12:47:11.551532 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:47:11.551962 7613 external_mini_cluster.cc:1428] Setting key f12259ded2f5bba593259388d77f817e
Aug 11 12:47:11 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452431, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM
I20260811 12:47:11.565433 18383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:11.548115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:55165 (local address 127.7.111.126:34613)
0811 12:47:11.548293 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:11.548298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:11.549529 (+ 1231us) server_negotiation.cc:407] Connection header received
0811 12:47:11.551387 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:11.551394 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:11.551490 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:11.551585 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:11.553327 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:11.554161 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:11.555804 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:11.556263 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:11.556376 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:11.559922 (+ 3546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:11.559927 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:11.559930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:11.559964 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:11.562446 (+ 2482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:11.563259 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:11.563263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:11.563265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:11.563389 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:11.563824 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:11.563826 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:11.563828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:11.564074 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:11.564190 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:11.564983 (+ 793us) server_negotiation.cc:299] Negotiation successful
0811 12:47:11.565149 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:47:11.566530 18583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:11.548602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.66:55165)
0811 12:47:11.549341 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:11.549413 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:11.551104 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:11.551863 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:11.551872 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:11.552439 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:11.553148 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:11.553169 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:11.554328 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:11.554332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:11.555648 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:11.555661 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:11.556494 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:11.556503 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:11.556726 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:11.557425 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:11.557450 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:11.559702 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:11.562600 (+ 2898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:11.562610 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:11.562625 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:11.563141 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:11.563482 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:11.563486 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:11.563489 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:11.563734 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:11.564199 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:11.564208 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:11.564617 (+ 409us) client_negotiation.cc:769] Sending connection context
0811 12:47:11.564927 (+ 310us) client_negotiation.cc:240] Negotiation successful
0811 12:47:11.565473 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":285,"threads_started":1}
I20260811 12:47:11.567885 18580 heartbeater.cc:347] Connected to a master server at 127.7.111.126:34613
I20260811 12:47:11.568210 18580 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:11.568950 18580 heartbeater.cc:511] Master 127.7.111.126:34613 requested a full tablet report, sending...
I20260811 12:47:11.570355 18089 ts_manager.cc:194] Registered new tserver with Master: 16939728527c4f3287b355862cc6a6f0 (127.7.111.66:46081)
I20260811 12:47:11.571127 18089 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:55165
WARNING: no policy specified for oryx/127.7.111.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.111.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.111.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.67.keytab.
Entry for principal oryx/127.7.111.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.67.keytab.
I20260811 12:47:11.606312 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34613
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.67.keytab
--principal=oryx/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:42611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:47:11.938997 18591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:11.939284 18591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:11.939422 18591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:11.946676 18591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:11.946791 18591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:11.946949 18591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:47:11.956440 18591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:42611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.67.keytab
--principal=oryx/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:11.957778 18591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:11.959111 18591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:11.971922 18601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:11.971995 18602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:11.971961 18606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:11.972754 18591 server_base.cc:1034] running on GCE node
I20260811 12:47:11.973417 18591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:11.974440 18591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:11.975680 18591 hybrid_clock.cc:648] HybridClock initialized: now 1786452431975645 us; error 87 us; skew 500 ppm
I20260811 12:47:11.976074 18591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:11 dist-test-slave-f0tn krb5kdc[18024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452431, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:11.979568 18591 init.cc:377] Logged in from keytab as oryx/127.7.111.67@KRBTEST.COM (short username oryx)
I20260811 12:47:11.981031 18591 webserver.cc:460] Webserver started at http://127.7.111.67:43635/ using document root <none> and password file <none>
I20260811 12:47:11.981818 18591 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:11.981940 18591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:11.982263 18591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:11.985308 18591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "9ae448d0f0be4eb3ac28cf4ff915a958"
format_stamp: "Formatted at 2026-08-11 12:47:11 on dist-test-slave-f0tn"
server_key: "e3d26f5ead72e95831c5dfb5a638232a"
server_key_iv: "78b54127d3b263e8eceab4dcbedeee93"
server_key_version: "encryptionkey@0"
I20260811 12:47:11.986110 18591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "9ae448d0f0be4eb3ac28cf4ff915a958"
format_stamp: "Formatted at 2026-08-11 12:47:11 on dist-test-slave-f0tn"
server_key: "e3d26f5ead72e95831c5dfb5a638232a"
server_key_iv: "78b54127d3b263e8eceab4dcbedeee93"
server_key_version: "encryptionkey@0"
I20260811 12:47:11.991596 18591 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:47:11.995316 18613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:11.996831 18591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:47:11.996974 18591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "9ae448d0f0be4eb3ac28cf4ff915a958"
format_stamp: "Formatted at 2026-08-11 12:47:11 on dist-test-slave-f0tn"
server_key: "e3d26f5ead72e95831c5dfb5a638232a"
server_key_iv: "78b54127d3b263e8eceab4dcbedeee93"
server_key_version: "encryptionkey@0"
I20260811 12:47:11.997153 18591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:12.025625 18591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:12.030174 18591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:12.030498 18591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:12.031512 18591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:12.032912 18591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:12.032994 18591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:12.033114 18591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:12.033176 18591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:12.064759 18591 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:39893
I20260811 12:47:12.064798 18781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:39893 every 8 connection(s)
I20260811 12:47:12.066040 18591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:47:12.069824 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 18591
I20260811 12:47:12.070031 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:47:12.070587 7613 external_mini_cluster.cc:1428] Setting key c9f845748758c3721beff59f8c120900
I20260811 12:47:12.076773 18381 heartbeater.cc:503] Master 127.7.111.126:34613 was elected leader, sending a full tablet report...
Aug 11 12:47:12 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452431, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM
I20260811 12:47:12.086933 18785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:12.068772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:48521 (local address 127.7.111.126:34613)
0811 12:47:12.069232 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:12.069236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:12.070393 (+ 1157us) server_negotiation.cc:407] Connection header received
0811 12:47:12.072353 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:12.072358 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:12.072438 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:12.072556 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:12.074474 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.075249 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.076842 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.077351 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.077500 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:12.081185 (+ 3685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:12.081189 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:12.081193 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:12.081225 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:12.083749 (+ 2524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:12.084478 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:12.084484 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:12.084487 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:12.084631 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:12.085121 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:12.085126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:12.085129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:12.085516 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:12.085669 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:12.086480 (+ 811us) server_negotiation.cc:299] Negotiation successful
0811 12:47:12.086655 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":223,"threads_started":1}
I20260811 12:47:12.087922 18787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:12.069359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.67:48521)
0811 12:47:12.070249 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:12.070288 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:12.072112 (+ 1824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:12.072878 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:12.072891 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:12.073679 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:12.074311 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:12.074330 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.075403 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.075408 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:12.076689 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:12.076704 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.077616 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.077625 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:12.077933 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:12.078673 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:12.078696 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:12.080954 (+ 2258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:12.083903 (+ 2949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:12.083911 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:12.083924 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:12.084358 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:12.084727 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:12.084732 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:12.084735 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:12.084988 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:12.085663 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:12.085670 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:12.086147 (+ 477us) client_negotiation.cc:769] Sending connection context
0811 12:47:12.086394 (+ 247us) client_negotiation.cc:240] Negotiation successful
0811 12:47:12.086799 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":323,"threads_started":1}
I20260811 12:47:12.089286 18782 heartbeater.cc:347] Connected to a master server at 127.7.111.126:34613
I20260811 12:47:12.089614 18782 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:12.090343 18782 heartbeater.cc:511] Master 127.7.111.126:34613 requested a full tablet report, sending...
I20260811 12:47:12.091826 18089 ts_manager.cc:194] Registered new tserver with Master: 9ae448d0f0be4eb3ac28cf4ff915a958 (127.7.111.67:39893)
I20260811 12:47:12.092640 18089 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:48521
I20260811 12:47:12.098359 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.111.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.111.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.111.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.125.keytab.
Entry for principal oryx/127.7.111.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.125.keytab.
I20260811 12:47:12.154217 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.125:38817
--webserver_interface=127.7.111.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.125.keytab
--principal=oryx/127.7.111.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:42611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:34613,127.7.111.125:38817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:47:12.500936 18793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:12.501240 18793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:12.501394 18793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:12.508512 18793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:47:12.508630 18793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:12.508718 18793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:12.508796 18793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:47:12.508865 18793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:47:12.520771 18793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:42611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:34613,127.7.111.125:38817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.125.keytab
--principal=oryx/127.7.111.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.125:38817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.111.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
W20260811 12:47:12.521322 18793 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.7.111.126:34613,127.7.111.125:38817'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 12:47:12.522711 18793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:12.524413 18793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:12.536357 18801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:12.536309 18805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:12.536276 18802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:12.537344 18793 server_base.cc:1034] running on GCE node
I20260811 12:47:12.538110 18793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:12.539543 18793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:12.540763 18793 hybrid_clock.cc:648] HybridClock initialized: now 1786452432540685 us; error 48 us; skew 500 ppm
I20260811 12:47:12.541069 18793 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:12 dist-test-slave-f0tn krb5kdc[18024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452432, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:12.544251 18793 init.cc:377] Logged in from keytab as oryx/127.7.111.125@KRBTEST.COM (short username oryx)
I20260811 12:47:12.545802 18793 webserver.cc:460] Webserver started at http://127.7.111.125:43921/ using document root <none> and password file <none>
I20260811 12:47:12.546460 18793 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:12.546545 18793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:12.546828 18793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:12.549767 18793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/data/instance:
uuid: "148ff30033374e829df5a9d57fb0ddba"
format_stamp: "Formatted at 2026-08-11 12:47:12 on dist-test-slave-f0tn"
server_key: "1f0d1be898867a7f8d85be2ec4cb5c2b"
server_key_iv: "3a43cc15f1bb3b44ad029ed9534bbe0a"
server_key_version: "encryptionkey@0"
I20260811 12:47:12.550480 18793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/wal/instance:
uuid: "148ff30033374e829df5a9d57fb0ddba"
format_stamp: "Formatted at 2026-08-11 12:47:12 on dist-test-slave-f0tn"
server_key: "1f0d1be898867a7f8d85be2ec4cb5c2b"
server_key_iv: "3a43cc15f1bb3b44ad029ed9534bbe0a"
server_key_version: "encryptionkey@0"
I20260811 12:47:12.555248 18793 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:47:12.558575 18817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:12.559787 18793 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:47:12.559944 18793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/wal
uuid: "148ff30033374e829df5a9d57fb0ddba"
format_stamp: "Formatted at 2026-08-11 12:47:12 on dist-test-slave-f0tn"
server_key: "1f0d1be898867a7f8d85be2ec4cb5c2b"
server_key_iv: "3a43cc15f1bb3b44ad029ed9534bbe0a"
server_key_version: "encryptionkey@0"
I20260811 12:47:12.560132 18793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:12.573454 18580 heartbeater.cc:503] Master 127.7.111.126:34613 was elected leader, sending a full tablet report...
I20260811 12:47:12.579088 18793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:12.583257 18793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:12.583498 18793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:12.601790 18793 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.125:38817
I20260811 12:47:12.601801 18897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.125:38817 every 8 connection(s)
I20260811 12:47:12.603235 18793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/data/info.pb
I20260811 12:47:12.607004 18901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:12.608979 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 18793
I20260811 12:47:12.614302 18901 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:47:12 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.111.125@KRBTEST.COM
Aug 11 12:47:12 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452432, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.125@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM
I20260811 12:47:12.631695 18906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:12.617177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42536 (local address 127.7.111.126:34613)
0811 12:47:12.617708 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:12.617714 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:12.618180 (+ 466us) server_negotiation.cc:407] Connection header received
0811 12:47:12.618379 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:12.618384 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:12.618462 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:12.618570 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:12.619623 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.620382 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.621862 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.622479 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.622614 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:12.626067 (+ 3453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:12.626071 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:12.626075 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:12.626118 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:12.628888 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:12.629681 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:12.629684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:12.629687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:12.629815 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:12.630206 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:12.630210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:12.630213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:12.630627 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:12.630768 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:12.631298 (+ 530us) server_negotiation.cc:299] Negotiation successful
0811 12:47:12.631457 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":241,"threads_started":1}
I20260811 12:47:12.632682 18905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:12.611332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60180 (local address 127.7.111.125:38817)
0811 12:47:12.612121 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:12.612132 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:12.612189 (+ 57us) server_negotiation.cc:407] Connection header received
0811 12:47:12.613887 (+ 1698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:12.613910 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:12.614284 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:12.614604 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:12.616417 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.618160 (+ 1743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.619519 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.620153 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.620286 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:12.624765 (+ 4479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:12.624773 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:12.624784 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:12.624818 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:12.628194 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:12.628875 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:12.628882 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:12.628890 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:12.629087 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:12.629488 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:12.629494 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:12.629501 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:12.630118 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:12.630358 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:12.630844 (+ 486us) server_negotiation.cc:299] Negotiation successful
0811 12:47:12.631269 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":337,"threads_started":1}
I20260811 12:47:12.632701 18908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:12.617210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.0.0.1:42536)
0811 12:47:12.618030 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:12.618072 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:12.618245 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:12.618611 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:12.618616 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:12.619220 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:12.619472 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:12.619484 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.620521 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.620524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:12.621709 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:12.621721 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.622665 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.622681 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:12.622812 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:12.623345 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:12.623367 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:12.625888 (+ 2521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:12.628998 (+ 3110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:12.629008 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:12.629024 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:12.629568 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:12.629911 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:12.629915 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:12.629918 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:12.630105 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:12.630754 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:12.630767 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:12.630940 (+ 173us) client_negotiation.cc:769] Sending connection context
0811 12:47:12.631197 (+ 257us) client_negotiation.cc:240] Negotiation successful
0811 12:47:12.631486 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":434,"threads_started":1}
I20260811 12:47:12.634603 18901 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:47:12 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452432, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.125@KRBTEST.COM for oryx/127.7.111.125@KRBTEST.COM
I20260811 12:47:12.646663 18908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:12.635554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:38817 (local address 127.0.0.1:60194)
0811 12:47:12.636094 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:12.636112 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:12.636206 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:12.636467 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:12.636472 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:12.636773 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:12.636902 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:12.636909 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.637861 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.637865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:12.638744 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:12.638755 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.639360 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.639365 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:12.639499 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:12.639937 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:12.639953 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:12.641933 (+ 1980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:12.644502 (+ 2569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:12.644507 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:12.644512 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:12.644861 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:12.645159 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:12.645163 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:12.645167 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:12.645296 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:12.645807 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:12.645812 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:12.645945 (+ 133us) client_negotiation.cc:769] Sending connection context
0811 12:47:12.646055 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 12:47:12.646314 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406}
I20260811 12:47:12.646760 18905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:12.635508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60194 (local address 127.7.111.125:38817)
0811 12:47:12.635693 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:12.635698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:12.636172 (+ 474us) server_negotiation.cc:407] Connection header received
0811 12:47:12.636296 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:12.636299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:12.636353 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:12.636429 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:12.637011 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.637729 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.638858 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.639237 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.639324 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:12.642106 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:12.642110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:12.642113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:12.642140 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:12.644387 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:12.644960 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:12.644964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:12.644966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:12.645065 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:12.645440 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:12.645445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:12.645448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:12.645693 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:12.645825 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:12.646148 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 12:47:12.646461 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":113,"server-negotiator.queue_time_us":56}
I20260811 12:47:12.652967 18901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba: Bootstrap starting.
I20260811 12:47:12.656200 18901 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:12.657274 18901 log.cc:824] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba: Log is configured to *not* fsync() on all Append() calls
I20260811 12:47:12.659838 18901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba: No bootstrap required, opened a new log
I20260811 12:47:12.666922 18901 raft_consensus.cc:348] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } }
I20260811 12:47:12.667218 18901 raft_consensus.cc:374] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:12.667290 18901 raft_consensus.cc:729] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148ff30033374e829df5a9d57fb0ddba, State: Initialized, Role: FOLLOWER
I20260811 12:47:12.667886 18901 consensus_queue.cc:260] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } }
I20260811 12:47:12.668684 18912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } } }
I20260811 12:47:12.668998 18912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:47:12.669657 18901 sys_catalog.cc:564] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [sys.catalog]: configured and running, proceeding with master startup.
W20260811 12:47:12.678213 18931 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:47:12.678366 18931 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:47:12.680104 18793 master_runner.cc:428] Detected that this master 148ff30033374e829df5a9d57fb0ddba is joining an existing cluster
I20260811 12:47:12.680192 18793 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.111.125:38817
I20260811 12:47:12.700519 18906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:12.683877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42550 (local address 127.7.111.126:34613)
0811 12:47:12.684060 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:12.684066 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:12.684717 (+ 651us) server_negotiation.cc:407] Connection header received
0811 12:47:12.684990 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:12.684996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:12.685090 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:12.685165 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:12.686773 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.687892 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.690120 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.690755 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.690852 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:12.693725 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:12.693731 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:12.693735 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:12.693772 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:12.697368 (+ 3596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:12.698269 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:12.698274 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:12.698277 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:12.698417 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:12.698926 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:12.698931 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:12.698934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:12.699282 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:12.699412 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:12.699961 (+ 549us) server_negotiation.cc:299] Negotiation successful
0811 12:47:12.700176 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:47:12.700544 18905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:12.684303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60198 (local address 127.7.111.125:38817)
0811 12:47:12.684496 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:12.684502 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:12.685311 (+ 809us) server_negotiation.cc:407] Connection header received
0811 12:47:12.685483 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:12.685488 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:12.685569 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:12.685696 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:12.686945 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.687979 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.690103 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.690784 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.690871 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:12.693709 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:12.693716 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:12.693720 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:12.693753 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:12.697372 (+ 3619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:12.698257 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:12.698261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:12.698264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:12.698421 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:12.698919 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:12.698923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:12.698926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:12.699263 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:12.699417 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:12.699955 (+ 538us) server_negotiation.cc:299] Negotiation successful
0811 12:47:12.700176 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:47:12.701581 18945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:12.684095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:38817 (local address 127.0.0.1:60198)
0811 12:47:12.685211 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:12.685228 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:12.685341 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:12.685728 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:12.685733 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:12.686476 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:12.686820 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:12.686833 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.688182 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.688186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:12.689980 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:12.689993 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.691143 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.691150 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:12.691349 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:12.692086 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:12.692112 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:12.693541 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:12.697538 (+ 3997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:12.697542 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:12.697546 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:12.698135 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:12.698598 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:12.698602 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:12.698605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:12.698798 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:12.699477 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:12.699482 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:12.699679 (+ 197us) client_negotiation.cc:769] Sending connection context
0811 12:47:12.700760 (+ 1081us) client_negotiation.cc:240] Negotiation successful
0811 12:47:12.701058 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":993,"mutex_wait_us":197,"spinlock_wait_cycles":51840,"thread_start_us":791,"threads_started":1}
I20260811 12:47:12.701678 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:12.683958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.0.0.1:42550)
0811 12:47:12.684619 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:12.684637 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:12.684830 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:12.685419 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:12.685424 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:12.686144 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:12.686576 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:12.686589 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.688076 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.688080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:12.689962 (+ 1882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:12.689977 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.691178 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.691185 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:12.691455 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:12.692186 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:12.692215 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:12.693511 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:12.697602 (+ 4091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:12.697606 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:12.697609 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:12.698129 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:12.698610 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:12.698617 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:12.698620 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:12.698808 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:12.699478 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:12.699482 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:12.699711 (+ 229us) client_negotiation.cc:769] Sending connection context
0811 12:47:12.700760 (+ 1049us) client_negotiation.cc:240] Negotiation successful
0811 12:47:12.701248 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"mutex_wait_us":266,"thread_start_us":535,"threads_started":1}
Aug 11 12:47:12 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452430, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.126@KRBTEST.COM for oryx/127.7.111.125@KRBTEST.COM
WARNING: no policy specified for oryx/127.7.111.124@KRBTEST.COM; defaulting to no policy
I20260811 12:47:12.721727 18905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:12.707437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60212 (local address 127.7.111.125:38817)
0811 12:47:12.707588 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:12.707593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:12.708314 (+ 721us) server_negotiation.cc:407] Connection header received
0811 12:47:12.708695 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:12.708711 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:12.708811 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:12.708887 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:12.710521 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.711225 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.712463 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.712931 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.713035 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:12.716162 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:12.716168 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:12.716173 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:12.716216 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:12.719004 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:12.719674 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:12.719680 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:12.719682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:12.719820 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:12.720186 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:12.720190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:12.720192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:12.720471 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:12.720599 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:12.721045 (+ 446us) server_negotiation.cc:299] Negotiation successful
0811 12:47:12.721362 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 12:47:12.721722 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:12.707589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:38817 (local address 127.0.0.1:60212)
0811 12:47:12.708197 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:12.708232 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:12.708449 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:12.709235 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:12.709242 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:12.710054 (+ 812us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:12.710350 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:12.710365 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.711348 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.711354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:12.712331 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:12.712342 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.713094 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.713100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:12.713274 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:12.713876 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:12.713894 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:12.715972 (+ 2078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:12.719127 (+ 3155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:12.719139 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:12.719147 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:12.719557 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:12.719925 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:12.719929 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:12.719933 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:12.720080 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:12.720595 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:12.720603 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:12.720773 (+ 170us) client_negotiation.cc:769] Sending connection context
0811 12:47:12.720933 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 12:47:12.721275 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":282,"threads_started":1}
I20260811 12:47:12.723909 18089 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.111.125:38817: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "148ff30033374e829df5a9d57fb0ddba"
member_type: NON_VOTER
last_known_addr {
host: "127.7.111.125"
port: 38817
}
attrs {
promote: true
}
}
dest_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506"
cas_config_opid_index: -1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.111.124@KRBTEST.COM" created.
I20260811 12:47:12.725093 18089 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: NON_VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: true } }
I20260811 12:47:12.739048 18951 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 148ff30033374e829df5a9d57fb0ddba (127.7.111.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: NON_VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: true } } }
I20260811 12:47:12.740338 18858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:12.743641 18951 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.111.125:38817
I20260811 12:47:12.744379 18858 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 1 FOLLOWER]: Refusing update from remote peer 8acd3cadf1424c2792ed3fc7e4c5f506: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 12:47:12.744591 18793 master.cc:537] Master@127.7.111.125:38817 shutting down...
I20260811 12:47:12.745756 18954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: NON_VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: true } } }
I20260811 12:47:12.746086 18954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:12.748075 18951 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [LEADER]: Connected to new peer: Peer: permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: NON_VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:47:12.760973 18793 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:47:12.761415 18793 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:47:12.761629 18793 tablet_replica.cc:330] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba: stopping tablet replica
W20260811 12:47:12.761554 18065 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260811 12:47:12.769199 18793 master.cc:552] Master@127.7.111.125:38817 shutdown complete.
I20260811 12:47:12.769290 18793 master_runner.cc:305] Clearing existing system tablet
I20260811 12:47:12.769851 18793 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:12.769975 18793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:12.773880 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:12.773332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:38817 (local address 127.0.0.1:60224)
0811 12:47:12.773566 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:12.773698 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:38817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260811 12:47:12.774111 18065 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 -> Peer 148ff30033374e829df5a9d57fb0ddba (127.7.111.125:38817): Couldn't send request to peer 148ff30033374e829df5a9d57fb0ddba. Status: Network error: Client connection negotiation failed: client connection to 127.7.111.125:38817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 12:47:12.774559 18793 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260811 12:47:12.776217 18962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:12.776621 18793 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:12.776880 18793 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260811 12:47:12.776969 18793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/wal
uuid: "148ff30033374e829df5a9d57fb0ddba"
format_stamp: "Formatted at 2026-08-11 12:47:12 on dist-test-slave-f0tn"
server_key: "1f0d1be898867a7f8d85be2ec4cb5c2b"
server_key_iv: "3a43cc15f1bb3b44ad029ed9534bbe0a"
server_key_version: "encryptionkey@0"
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.111.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.124.keytab.
Entry for principal oryx/127.7.111.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.124.keytab.
I20260811 12:47:12.780645 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.124:44773
--webserver_interface=127.7.111.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.124.keytab
--principal=oryx/127.7.111.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:42611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:34613,127.7.111.125:38817,127.7.111.124:44773
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:47:12.782658 18793 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:47:12.789685 18793 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 12:47:12.789870 18793 log.cc:1196] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 12:47:12.790313 18793 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba: Deleting consensus metadata
I20260811 12:47:12.790598 18793 master_runner.cc:315] Copying system tablet from 127.7.111.126:34613
I20260811 12:47:12.796293 18793 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba: tablet copy: Beginning tablet copy session from remote peer at address 127.7.111.126:34613
I20260811 12:47:12.812201 18906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:12.797168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42552 (local address 127.7.111.126:34613)
0811 12:47:12.797358 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:12.797364 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:12.797865 (+ 501us) server_negotiation.cc:407] Connection header received
0811 12:47:12.798136 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:12.798141 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:12.798228 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:12.798328 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:12.799427 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.800512 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.802372 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.803021 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.803151 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:12.805516 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:12.805523 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:12.805528 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:12.805571 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:12.809063 (+ 3492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:12.809913 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:12.809918 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:12.809922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:12.810070 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:12.810580 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:12.810586 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:12.810590 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:12.810961 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:12.811117 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:12.811596 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 12:47:12.811840 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:47:12.812222 18974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:12.797093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.0.0.1:42552)
0811 12:47:12.797751 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:12.797771 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:12.797931 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:12.798379 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:12.798384 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:12.798947 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:12.799246 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:12.799259 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.800692 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.800699 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:12.802210 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:12.802226 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:12.803210 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:12.803217 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:12.803395 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:12.804138 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:12.804161 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:12.805271 (+ 1110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:12.809201 (+ 3930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:12.809207 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:12.809211 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:12.809805 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:12.810190 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:12.810198 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:12.810202 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:12.810412 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:12.811112 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:12.811118 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:12.811310 (+ 192us) client_negotiation.cc:769] Sending connection context
0811 12:47:12.811451 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 12:47:12.811775 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"spinlock_wait_cycles":7552,"thread_start_us":491,"threads_started":1}
I20260811 12:47:12.813428 18120 tablet_copy_service.cc:140] P 8acd3cadf1424c2792ed3fc7e4c5f506: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 148ff30033374e829df5a9d57fb0ddba ({username='oryx', principal='oryx/127.7.111.125@KRBTEST.COM'} at 127.0.0.1:42552)
I20260811 12:47:12.813750 18120 tablet_copy_service.cc:161] P 8acd3cadf1424c2792ed3fc7e4c5f506: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 148ff30033374e829df5a9d57fb0ddba at {username='oryx', principal='oryx/127.7.111.125@KRBTEST.COM'} at 127.0.0.1:42552: session id = 148ff30033374e829df5a9d57fb0ddba-00000000000000000000000000000000
I20260811 12:47:12.818481 18120 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 12:47:12.820968 18793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 12:47:12.828032 18793 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba: tablet copy: Starting download of 0 data blocks...
I20260811 12:47:12.828359 18793 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba: tablet copy: Starting download of 1 WAL segments...
I20260811 12:47:12.831499 18793 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 12:47:12.836144 18120 tablet_copy_service.cc:342] P 8acd3cadf1424c2792ed3fc7e4c5f506: Request end of tablet copy session 148ff30033374e829df5a9d57fb0ddba-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.111.125@KRBTEST.COM'} at 127.0.0.1:42552
I20260811 12:47:12.836375 18120 tablet_copy_service.cc:433] P 8acd3cadf1424c2792ed3fc7e4c5f506: ending tablet copy session 148ff30033374e829df5a9d57fb0ddba-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 148ff30033374e829df5a9d57fb0ddba
I20260811 12:47:12.849517 18793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:12.852500 18976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:12.853307 18978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:12.857843 18793 server_base.cc:1034] running on GCE node
W20260811 12:47:12.858235 18982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:12.858681 18793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:12.859334 18793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:12.860543 18793 hybrid_clock.cc:648] HybridClock initialized: now 1786452432860488 us; error 28 us; skew 500 ppm
I20260811 12:47:12.860774 18793 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:12 dist-test-slave-f0tn krb5kdc[18024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452432, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:12.863062 18793 init.cc:377] Logged in from keytab as oryx/127.7.111.125@KRBTEST.COM (short username oryx)
I20260811 12:47:12.864331 18793 webserver.cc:460] Webserver started at http://127.7.111.125:41387/ using document root <none> and password file <none>
I20260811 12:47:12.864763 18793 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:12.864881 18793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:12.868115 18793 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.002s
I20260811 12:47:12.870203 18994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:12.870625 18793 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260811 12:47:12.870774 18793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/wal
uuid: "148ff30033374e829df5a9d57fb0ddba"
format_stamp: "Formatted at 2026-08-11 12:47:12 on dist-test-slave-f0tn"
server_key: "1f0d1be898867a7f8d85be2ec4cb5c2b"
server_key_iv: "3a43cc15f1bb3b44ad029ed9534bbe0a"
server_key_version: "encryptionkey@0"
I20260811 12:47:12.870945 18793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:12.891986 18793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:12.896073 18793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:12.918963 18793 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.125:38817
I20260811 12:47:12.918988 19078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.125:38817 every 8 connection(s)
I20260811 12:47:12.920013 18793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-1/data/info.pb
I20260811 12:47:12.925414 19079 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:47:12.929199 19079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba: Bootstrap starting.
I20260811 12:47:12.947192 19079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:47:12.947844 19079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba: Bootstrap complete.
I20260811 12:47:12.949002 19079 raft_consensus.cc:348] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: NON_VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: true } }
I20260811 12:47:12.949290 19079 raft_consensus.cc:729] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 148ff30033374e829df5a9d57fb0ddba, State: Initialized, Role: LEARNER
I20260811 12:47:12.949615 19079 consensus_queue.cc:260] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: NON_VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: true } }
I20260811 12:47:12.950515 19085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: NON_VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: true } } }
I20260811 12:47:12.950788 19079 sys_catalog.cc:564] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:47:12.950922 19085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [sys.catalog]: This master's current role is: LEARNER
I20260811 12:47:12.958422 19106 catalog_manager.cc:1259] Loaded cluster ID: 53e0569ae84d4f13b3a2305a0117f25e
I20260811 12:47:12.958616 19106 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba: loading cluster ID for follower catalog manager: success
I20260811 12:47:12.962050 19106 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba: acquiring CA information for follower catalog manager: success
I20260811 12:47:12.964682 19106 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:47:13.095197 18782 heartbeater.cc:503] Master 127.7.111.126:34613 was elected leader, sending a full tablet report...
I20260811 12:47:13.148212 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:13.141239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:38817 (local address 127.0.0.1:60238)
0811 12:47:13.141500 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:13.141530 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:13.141672 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:13.142453 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:13.142459 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:13.142491 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:47:13.142713 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:13.142726 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.144137 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.144143 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:13.146135 (+ 1992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:13.146152 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.147335 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.147342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:13.147591 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:13.147658 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 12:47:13.147754 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 12:47:13.147900 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 12:47:13.148316 19107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:13.141525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60238 (local address 127.7.111.125:38817)
0811 12:47:13.142115 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:13.142119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:13.142137 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:47:13.142201 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:13.142204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:13.142272 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:13.142373 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:47:13.142979 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.143984 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.146271 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.147152 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.147257 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:13.147442 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:13.147807 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 12:47:13.147949 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":332,"threads_started":1}
I20260811 12:47:13.149833 19043 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260811 12:47:13.150475 19085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8acd3cadf1424c2792ed3fc7e4c5f506. Latest consensus state: current_term: 1 leader_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: NON_VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: true } } }
I20260811 12:47:13.150755 19085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [sys.catalog]: This master's current role is: LEARNER
W20260811 12:47:13.163797 18963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:13.164122 18963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:13.164265 18963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:13.171587 18963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:47:13.171680 18963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:13.171761 18963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:13.171828 18963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:47:13.171883 18963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:47:13.181638 18963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:42611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:34613,127.7.111.125:38817,127.7.111.124:44773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.124.keytab
--principal=oryx/127.7.111.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.124:44773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.111.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:13.182940 18963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:13.184259 18963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:13.195575 19117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:13.195605 19121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:13.195648 19119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:13.196954 18963 server_base.cc:1034] running on GCE node
I20260811 12:47:13.197867 18963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:13.199347 18963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:13.200536 18963 hybrid_clock.cc:648] HybridClock initialized: now 1786452433200483 us; error 48 us; skew 500 ppm
I20260811 12:47:13.200876 18963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:13 dist-test-slave-f0tn krb5kdc[18024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452433, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:13.203923 18963 init.cc:377] Logged in from keytab as oryx/127.7.111.124@KRBTEST.COM (short username oryx)
I20260811 12:47:13.205322 18963 webserver.cc:460] Webserver started at http://127.7.111.124:39513/ using document root <none> and password file <none>
I20260811 12:47:13.206023 18963 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:13.206144 18963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:13.206441 18963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:13.209432 18963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/data/instance:
uuid: "88bd75212b2e4308a018df055d5b6ccd"
format_stamp: "Formatted at 2026-08-11 12:47:13 on dist-test-slave-f0tn"
server_key: "7c3bb2c2a9f5fa651dbd9d0f463fdd17"
server_key_iv: "faff1bd1ff5b9039b8cc4e38da170194"
server_key_version: "encryptionkey@0"
I20260811 12:47:13.210222 18963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/wal/instance:
uuid: "88bd75212b2e4308a018df055d5b6ccd"
format_stamp: "Formatted at 2026-08-11 12:47:13 on dist-test-slave-f0tn"
server_key: "7c3bb2c2a9f5fa651dbd9d0f463fdd17"
server_key_iv: "faff1bd1ff5b9039b8cc4e38da170194"
server_key_version: "encryptionkey@0"
I20260811 12:47:13.215317 18963 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:47:13.218724 19132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:13.219954 18963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:47:13.220110 18963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/wal
uuid: "88bd75212b2e4308a018df055d5b6ccd"
format_stamp: "Formatted at 2026-08-11 12:47:13 on dist-test-slave-f0tn"
server_key: "7c3bb2c2a9f5fa651dbd9d0f463fdd17"
server_key_iv: "faff1bd1ff5b9039b8cc4e38da170194"
server_key_version: "encryptionkey@0"
I20260811 12:47:13.220290 18963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:13.233713 18963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:13.237810 18963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:13.238061 18963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:13.259119 18963 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.124:44773
I20260811 12:47:13.259145 19223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.124:44773 every 8 connection(s)
I20260811 12:47:13.260545 18963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/data/info.pb
I20260811 12:47:13.264381 19224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:13.268455 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 18963
I20260811 12:47:13.271852 19224 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:47:13 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.111.124@KRBTEST.COM
Aug 11 12:47:13 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452433, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.124@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM
I20260811 12:47:13.291523 18906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:13.276460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42562 (local address 127.7.111.126:34613)
0811 12:47:13.276623 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:13.276628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:13.277033 (+ 405us) server_negotiation.cc:407] Connection header received
0811 12:47:13.277190 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:13.277196 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:13.277286 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:13.277450 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:13.278831 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.279803 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.281575 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.282033 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.282130 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:13.285489 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:13.285495 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:13.285500 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:13.285541 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:13.288529 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:13.289277 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:13.289281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:13.289283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:13.289453 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:13.289824 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:13.289828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:13.289831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:13.290148 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:13.290306 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:13.290933 (+ 627us) server_negotiation.cc:299] Negotiation successful
0811 12:47:13.291162 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 12:47:13.292582 19227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:13.270695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52980 (local address 127.7.111.124:44773)
0811 12:47:13.271582 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:13.271594 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:13.271636 (+ 42us) server_negotiation.cc:407] Connection header received
0811 12:47:13.273271 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:13.273300 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:13.273838 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:13.274379 (+ 541us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:13.275633 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.277373 (+ 1740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.279070 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.280047 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.280180 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:13.284963 (+ 4783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:13.284971 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:13.284988 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:13.285023 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:13.288232 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:13.288912 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:13.288919 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:13.288921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:13.289079 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:13.289505 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:13.289510 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:13.289513 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:13.290180 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:13.290437 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:13.290836 (+ 399us) server_negotiation.cc:299] Negotiation successful
0811 12:47:13.291476 (+ 640us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":664,"thread_start_us":449,"threads_started":1}
I20260811 12:47:13.292618 19229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:13.276185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.0.0.1:42562)
0811 12:47:13.276867 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:13.276919 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:13.277072 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:13.277481 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:13.277486 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:13.278340 (+ 854us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:13.278648 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:13.278660 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.279954 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.279958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:13.281434 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:13.281455 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.282256 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.282263 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:13.282447 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:13.283043 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:13.283073 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:13.285285 (+ 2212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:13.288647 (+ 3362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:13.288660 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:13.288672 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:13.289170 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:13.289559 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:13.289562 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:13.289564 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:13.289720 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:13.290304 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:13.290311 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:13.290532 (+ 221us) client_negotiation.cc:769] Sending connection context
0811 12:47:13.290834 (+ 302us) client_negotiation.cc:240] Negotiation successful
0811 12:47:13.291575 (+ 741us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":335,"thread_start_us":288,"threads_started":1}
I20260811 12:47:13.294478 19224 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:47:13 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452433, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.124@KRBTEST.COM for oryx/127.7.111.125@KRBTEST.COM
I20260811 12:47:13.306226 19229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:13.295363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:38817 (local address 127.0.0.1:60250)
0811 12:47:13.295647 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:13.295661 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:13.295745 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:13.296024 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:13.296027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:13.296348 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:13.296475 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:13.296483 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.297449 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.297453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:13.298403 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:13.298415 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.299125 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.299129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:13.299277 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:13.299766 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:13.299782 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:13.301705 (+ 1923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:13.304308 (+ 2603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:13.304312 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:13.304315 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:13.304664 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:13.304952 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:13.304955 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:13.304957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:13.305074 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:13.305535 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:13.305538 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:13.305657 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 12:47:13.305767 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 12:47:13.305971 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184}
I20260811 12:47:13.306252 19107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:13.295340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60250 (local address 127.7.111.125:38817)
0811 12:47:13.295592 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:13.295597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:13.295701 (+ 104us) server_negotiation.cc:407] Connection header received
0811 12:47:13.295841 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:13.295847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:13.295925 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:13.296004 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:13.296594 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.297308 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.298520 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.299012 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.299100 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:13.301879 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:13.301883 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:13.301886 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:13.301918 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:13.304196 (+ 2278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:13.304754 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:13.304758 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:13.304760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:13.304878 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:13.305145 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:13.305148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:13.305149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:13.305437 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:13.305551 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:13.305843 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 12:47:13.306018 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260811 12:47:13.307274 19224 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:47:13 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452433, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.124@KRBTEST.COM for oryx/127.7.111.124@KRBTEST.COM
I20260811 12:47:13.318190 19229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:13.307979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:44773 (local address 127.0.0.1:52986)
0811 12:47:13.308130 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:13.308145 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:13.308228 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:13.308449 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:13.308452 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:13.308761 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:13.308891 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:13.308898 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.309811 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.309814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:13.310631 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:13.310642 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.311269 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.311272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:13.311416 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:13.311918 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:13.311933 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:13.313736 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:13.316223 (+ 2487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:13.316226 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:13.316229 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:13.316557 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:13.316867 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:13.316870 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:13.316872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:13.317029 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:13.317469 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:13.317472 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:13.317586 (+ 114us) client_negotiation.cc:769] Sending connection context
0811 12:47:13.317705 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 12:47:13.317885 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 12:47:13.318290 19227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:13.308033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52986 (local address 127.7.111.124:44773)
0811 12:47:13.308185 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:13.308189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:13.308204 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:47:13.308308 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:13.308311 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:13.308362 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:13.308424 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:13.309000 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.309709 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.310748 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.311152 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.311243 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:13.313896 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:13.313900 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:13.313903 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:13.313932 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:13.316102 (+ 2170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:13.316648 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:13.316651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:13.316653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:13.316780 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:13.317112 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:13.317115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:13.317117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:13.317358 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:13.317497 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:13.317795 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 12:47:13.317991 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":56}
I20260811 12:47:13.323028 19224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd: Bootstrap starting.
I20260811 12:47:13.326193 19224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:13.327211 19224 log.cc:824] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd: Log is configured to *not* fsync() on all Append() calls
I20260811 12:47:13.329814 19224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd: No bootstrap required, opened a new log
I20260811 12:47:13.336828 19224 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } }
I20260811 12:47:13.337137 19224 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:13.337253 19224 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88bd75212b2e4308a018df055d5b6ccd, State: Initialized, Role: FOLLOWER
I20260811 12:47:13.337848 19224 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } }
I20260811 12:47:13.338742 19233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } } }
I20260811 12:47:13.339071 19233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:47:13.339742 19224 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:47:13.347477 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 18177
W20260811 12:47:13.349869 19251 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:47:13.350000 19251 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:47:13.353751 18963 master_runner.cc:428] Detected that this master 88bd75212b2e4308a018df055d5b6ccd is joining an existing cluster
I20260811 12:47:13.353864 18963 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.111.124:44773
Aug 11 12:47:13 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.111.65@KRBTEST.COM
I20260811 12:47:13.371254 19253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:13.349248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55390 (local address 127.7.111.65:33335)
0811 12:47:13.350550 (+ 1302us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:13.350558 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:13.350586 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:47:13.350654 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:13.350662 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:13.350876 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:13.351038 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:13.352769 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.354244 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.356091 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.356986 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.357111 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:13.362884 (+ 5773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:13.362901 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:13.362919 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:13.362966 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:13.366878 (+ 3912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:13.367792 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:13.367803 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:13.367807 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:13.367969 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:13.368471 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:13.368476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:13.368480 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:13.368936 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:13.369257 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:13.370540 (+ 1283us) server_negotiation.cc:299] Negotiation successful
0811 12:47:13.370820 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1145,"thread_start_us":984,"threads_started":1}
I20260811 12:47:13.373306 18373 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:13.379492 19261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:13.358767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:44773 (local address 127.0.0.1:52988)
0811 12:47:13.359527 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:13.359546 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:13.359739 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:13.360222 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:13.360228 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:13.360984 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:13.361367 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:13.361406 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.362961 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.362967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:13.364634 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:13.364652 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.366363 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.366374 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:13.366574 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:13.367255 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:13.367282 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:13.368507 (+ 1225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:13.372796 (+ 4289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:13.372801 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:13.372805 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:13.376899 (+ 4094us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:13.377363 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:13.377367 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:13.377371 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:13.377571 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:13.378302 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:13.378306 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:13.378502 (+ 196us) client_negotiation.cc:769] Sending connection context
0811 12:47:13.378693 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 12:47:13.379003 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":453,"threads_started":1}
I20260811 12:47:13.380399 19227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:13.358646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52988 (local address 127.7.111.124:44773)
0811 12:47:13.358853 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:13.358860 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:13.359618 (+ 758us) server_negotiation.cc:407] Connection header received
0811 12:47:13.359881 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:13.359886 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:13.359954 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:13.360025 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:13.361542 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.362741 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.364793 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.365565 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.365655 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:13.368708 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:13.368714 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:13.368718 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:13.368754 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:13.372591 (+ 3837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:13.377035 (+ 4444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:13.377044 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:13.377047 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:13.377222 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:13.377709 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:13.377714 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:13.377717 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:13.378078 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:13.378249 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:13.379858 (+ 1609us) server_negotiation.cc:299] Negotiation successful
0811 12:47:13.380080 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 12:47:13.393316 18906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:13.359880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42566 (local address 127.7.111.126:34613)
0811 12:47:13.360051 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:13.360057 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:13.377606 (+ 17549us) server_negotiation.cc:407] Connection header received
0811 12:47:13.377896 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:13.377903 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:13.377988 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:13.378057 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:13.379518 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.380657 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.383118 (+ 2461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.383770 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.383869 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:13.386752 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:13.386759 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:13.386764 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:13.386806 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:13.390155 (+ 3349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:13.390982 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:13.390988 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:13.390992 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:13.391140 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:13.391582 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:13.391587 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:13.391591 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:13.391958 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:13.392092 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:13.392686 (+ 594us) server_negotiation.cc:299] Negotiation successful
0811 12:47:13.392939 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:47:13.394403 19262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:13.359925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.0.0.1:42566)
0811 12:47:13.377492 (+ 17567us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:13.377512 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:13.377725 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:13.378272 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:13.378277 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:13.379005 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:13.379326 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:13.379340 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.380881 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.380886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:13.382939 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:13.382955 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.384176 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.384182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:13.384355 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:13.385313 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:13.385336 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:13.386524 (+ 1188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:13.390330 (+ 3806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:13.390338 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:13.390343 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:13.390842 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:13.391277 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:13.391285 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:13.391289 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:13.391460 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:13.392175 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:13.392181 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:13.392386 (+ 205us) client_negotiation.cc:769] Sending connection context
0811 12:47:13.393604 (+ 1218us) client_negotiation.cc:240] Negotiation successful
0811 12:47:13.393952 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17401,"thread_start_us":25457,"threads_started":1}
I20260811 12:47:13.398382 19107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:13.365966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60262 (local address 127.7.111.125:38817)
0811 12:47:13.366150 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:13.366156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:13.382616 (+ 16460us) server_negotiation.cc:407] Connection header received
0811 12:47:13.382886 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:13.382892 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:13.382975 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:13.383048 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:13.384256 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.385332 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.387308 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.387970 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.388070 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:13.391562 (+ 3492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:13.391569 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:13.391574 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:13.391620 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:13.395089 (+ 3469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:13.395967 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:13.395973 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:13.395977 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:13.396145 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:13.396636 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:13.396641 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:13.396644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:13.397014 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:13.397138 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:13.397785 (+ 647us) server_negotiation.cc:299] Negotiation successful
0811 12:47:13.398046 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 12:47:13.399336 19261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:13.366595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:38817 (local address 127.0.0.1:60262)
0811 12:47:13.382525 (+ 15930us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:13.382545 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:13.382716 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:13.383260 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:13.383265 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:13.383789 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:13.384054 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:13.384072 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.385569 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.385573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:13.387123 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:13.387139 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.388322 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.388328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:13.388510 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:13.389172 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:13.389195 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:13.391341 (+ 2146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:13.395270 (+ 3929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:13.395276 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:13.395279 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:13.395824 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:13.396311 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:13.396315 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:13.396319 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:13.396498 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:13.397245 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:13.397251 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:13.397433 (+ 182us) client_negotiation.cc:769] Sending connection context
0811 12:47:13.398645 (+ 1212us) client_negotiation.cc:240] Negotiation successful
0811 12:47:13.398985 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15821,"thread_start_us":26791,"threads_started":1}
Aug 11 12:47:13 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452430, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.126@KRBTEST.COM for oryx/127.7.111.124@KRBTEST.COM
I20260811 12:47:13.416602 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:13.399949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:44773 (local address 127.0.0.1:52998)
0811 12:47:13.400164 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:13.400185 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:13.400293 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:13.401470 (+ 1177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:13.401475 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:13.401983 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:13.402211 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:13.402225 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.403590 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.403595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:13.405076 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:13.405093 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.406187 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.406194 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:13.406417 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:13.407003 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:13.407030 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:13.409879 (+ 2849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:13.413923 (+ 4044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:13.413928 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:13.413933 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:13.414415 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:13.414886 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:13.414892 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:13.414896 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:13.415073 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:13.415688 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:13.415693 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:13.415858 (+ 165us) client_negotiation.cc:769] Sending connection context
0811 12:47:13.415997 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 12:47:13.416263 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 12:47:13.416904 19227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:13.400870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52998 (local address 127.7.111.124:44773)
0811 12:47:13.401044 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:13.401050 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:13.401066 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:47:13.401162 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:13.401167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:13.401280 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:13.401393 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:13.402375 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.403448 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.405238 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.406023 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.406145 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:13.410084 (+ 3939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:13.410091 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:13.410096 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:13.410134 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:13.413772 (+ 3638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:13.414579 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:13.414584 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:13.414588 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:13.414751 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:13.415199 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:13.415205 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:13.415208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:13.415574 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:13.415741 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:13.416109 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 12:47:13.416331 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 12:47:13.418390 18089 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.111.124:44773: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd"
member_type: NON_VOTER
last_known_addr {
host: "127.7.111.124"
port: 44773
}
attrs {
promote: true
}
}
dest_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506"
cas_config_opid_index: 5
I20260811 12:47:13.419476 18089 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: NON_VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: true } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: NON_VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: true } }
I20260811 12:47:13.422204 19043 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 1 LEARNER]: Refusing update from remote peer 8acd3cadf1424c2792ed3fc7e4c5f506: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 12:47:13.423813 19172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:13.425526 18951 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 88bd75212b2e4308a018df055d5b6ccd (127.7.111.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: NON_VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: true } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: NON_VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: true } } }
I20260811 12:47:13.429242 18951 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.111.124:44773
I20260811 12:47:13.431888 19172 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 1 FOLLOWER]: Refusing update from remote peer 8acd3cadf1424c2792ed3fc7e4c5f506: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 12:47:13.432447 18963 master.cc:537] Master@127.7.111.124:44773 shutting down...
I20260811 12:47:13.433678 19266 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [LEADER]: Connected to new peer: Peer: permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: NON_VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 12:47:13.434094 18951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: NON_VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: true } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: NON_VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: true } } }
I20260811 12:47:13.434617 18951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:13.434396 19266 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: NON_VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:47:13.442283 19043 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 88bd75212b2e4308a018df055d5b6ccd (127.7.111.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: NON_VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: true } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: NON_VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: true } } }
I20260811 12:47:13.446382 19085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: NON_VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: true } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: NON_VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: true } } }
I20260811 12:47:13.446659 19085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [sys.catalog]: This master's current role is: LEARNER
I20260811 12:47:13.447116 19266 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506: attempting to promote NON_VOTER 148ff30033374e829df5a9d57fb0ddba to VOTER
I20260811 12:47:13.447750 19266 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: NON_VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: true } }
I20260811 12:47:13.449579 19043 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 1 LEARNER]: Refusing update from remote peer 8acd3cadf1424c2792ed3fc7e4c5f506: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 12:47:13.450084 19266 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [LEADER]: Connected to new peer: Peer: permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 12:47:13.452195 19270 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 148ff30033374e829df5a9d57fb0ddba (127.7.111.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: NON_VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: true } } }
I20260811 12:47:13.455420 19266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: NON_VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: true } } }
I20260811 12:47:13.455663 19266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:13.455945 19270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: NON_VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: true } } }
I20260811 12:47:13.456152 19270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:13.456692 19043 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 148ff30033374e829df5a9d57fb0ddba (127.7.111.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: NON_VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: true } } }
I20260811 12:47:13.459537 19085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: NON_VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: true } } }
I20260811 12:47:13.459794 19085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:47:13.468000 18963 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:47:13.468469 18963 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:47:13.468695 18963 tablet_replica.cc:330] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd: stopping tablet replica
I20260811 12:47:13.477687 18963 master.cc:552] Master@127.7.111.124:44773 shutdown complete.
I20260811 12:47:13.477785 18963 master_runner.cc:305] Clearing existing system tablet
I20260811 12:47:13.478245 18963 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:13.478364 18963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:13.484077 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:13.483573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:44773 (local address 127.0.0.1:53006)
0811 12:47:13.483767 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:13.483871 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:44773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 12:47:13.484529 18062 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 -> Peer 88bd75212b2e4308a018df055d5b6ccd (127.7.111.124:44773): Couldn't send request to peer 88bd75212b2e4308a018df055d5b6ccd. Status: Network error: Client connection negotiation failed: client connection to 127.7.111.124:44773: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 12:47:13.485435 18963 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.005s
I20260811 12:47:13.486850 19276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:13.487164 18963 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:13.487404 18963 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 12:47:13.487509 18963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/wal
uuid: "88bd75212b2e4308a018df055d5b6ccd"
format_stamp: "Formatted at 2026-08-11 12:47:13 on dist-test-slave-f0tn"
server_key: "7c3bb2c2a9f5fa651dbd9d0f463fdd17"
server_key_iv: "faff1bd1ff5b9039b8cc4e38da170194"
server_key_version: "encryptionkey@0"
I20260811 12:47:13.490917 18963 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:47:13.495508 18963 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 12:47:13.495661 18963 log.cc:1196] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 12:47:13.496053 18963 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd: Deleting consensus metadata
I20260811 12:47:13.496292 18963 master_runner.cc:315] Copying system tablet from 127.7.111.126:34613
I20260811 12:47:13.502018 18963 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd: tablet copy: Beginning tablet copy session from remote peer at address 127.7.111.126:34613
I20260811 12:47:13.518272 18906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:13.502752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42576 (local address 127.7.111.126:34613)
0811 12:47:13.502993 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:13.503001 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:13.505907 (+ 2906us) server_negotiation.cc:407] Connection header received
0811 12:47:13.506084 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:13.506089 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:13.506158 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:13.506247 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:13.507122 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.507860 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.509124 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.509630 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.509771 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:13.511573 (+ 1802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:13.511580 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:13.511585 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:13.511623 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:13.515093 (+ 3470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:13.515731 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:13.515736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:13.515739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:13.515898 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:13.516514 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:13.516518 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:13.516522 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:13.516886 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:13.517015 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:13.517671 (+ 656us) server_negotiation.cc:299] Negotiation successful
0811 12:47:13.517910 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260811 12:47:13.519326 19284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:13.502744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.0.0.1:42576)
0811 12:47:13.505794 (+ 3050us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:13.505811 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:13.505959 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:13.506297 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:13.506301 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:13.506760 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:13.506963 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:13.506974 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.508006 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.508010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:13.508973 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:13.508987 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:13.509804 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:13.509811 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:13.509971 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:13.510581 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:13.510599 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:13.511345 (+ 746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:13.515231 (+ 3886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:13.515237 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:13.515241 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:13.515614 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:13.516133 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:13.516138 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:13.516142 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:13.516359 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:13.517121 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:13.517126 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:13.517329 (+ 203us) client_negotiation.cc:769] Sending connection context
0811 12:47:13.518529 (+ 1200us) client_negotiation.cc:240] Negotiation successful
0811 12:47:13.518864 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2921,"thread_start_us":2791,"threads_started":1}
I20260811 12:47:13.519944 18120 tablet_copy_service.cc:140] P 8acd3cadf1424c2792ed3fc7e4c5f506: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 88bd75212b2e4308a018df055d5b6ccd ({username='oryx', principal='oryx/127.7.111.124@KRBTEST.COM'} at 127.0.0.1:42576)
I20260811 12:47:13.520131 18120 tablet_copy_service.cc:161] P 8acd3cadf1424c2792ed3fc7e4c5f506: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 88bd75212b2e4308a018df055d5b6ccd at {username='oryx', principal='oryx/127.7.111.124@KRBTEST.COM'} at 127.0.0.1:42576: session id = 88bd75212b2e4308a018df055d5b6ccd-00000000000000000000000000000000
I20260811 12:47:13.525298 18120 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 12:47:13.527293 18963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 12:47:13.533273 18963 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd: tablet copy: Starting download of 0 data blocks...
I20260811 12:47:13.533610 18963 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd: tablet copy: Starting download of 1 WAL segments...
I20260811 12:47:13.536060 18963 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 12:47:13.539690 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 18177
I20260811 12:47:13.550565 18120 tablet_copy_service.cc:342] P 8acd3cadf1424c2792ed3fc7e4c5f506: Request end of tablet copy session 88bd75212b2e4308a018df055d5b6ccd-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.111.124@KRBTEST.COM'} at 127.0.0.1:42576
I20260811 12:47:13.550734 18120 tablet_copy_service.cc:433] P 8acd3cadf1424c2792ed3fc7e4c5f506: ending tablet copy session 88bd75212b2e4308a018df055d5b6ccd-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 88bd75212b2e4308a018df055d5b6ccd
I20260811 12:47:13.570439 18963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:13.573163 19285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:13.578756 19286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:13.583320 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:33335
--local_ip_for_outbound_sockets=127.7.111.65
--tserver_master_addrs=127.7.111.126:34613
--webserver_port=43627
--webserver_interface=127.7.111.65
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.65.keytab
--principal=oryx/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:42611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.111.126:34613,127.7.111.125:38817
--tserver_master_addrs=127.7.111.126:34613,127.7.111.125:38817,127.7.111.124:44773 with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:47:13.589634 18963 server_base.cc:1034] running on GCE node
W20260811 12:47:13.589553 19289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:13.590291 18963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:13.591182 18963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:13.592442 18963 hybrid_clock.cc:648] HybridClock initialized: now 1786452433592396 us; error 33 us; skew 500 ppm
I20260811 12:47:13.592680 18963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:13 dist-test-slave-f0tn krb5kdc[18024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452433, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:13.594831 18963 init.cc:377] Logged in from keytab as oryx/127.7.111.124@KRBTEST.COM (short username oryx)
I20260811 12:47:13.596186 18963 webserver.cc:460] Webserver started at http://127.7.111.124:42891/ using document root <none> and password file <none>
I20260811 12:47:13.596495 18963 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:13.596597 18963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:13.600945 18963 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:47:13.603134 19302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:13.603609 18963 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 12:47:13.603791 18963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/wal
uuid: "88bd75212b2e4308a018df055d5b6ccd"
format_stamp: "Formatted at 2026-08-11 12:47:13 on dist-test-slave-f0tn"
server_key: "7c3bb2c2a9f5fa651dbd9d0f463fdd17"
server_key_iv: "faff1bd1ff5b9039b8cc4e38da170194"
server_key_version: "encryptionkey@0"
I20260811 12:47:13.604053 18963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:13.627214 18963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:13.631139 18963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:13.668267 18963 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.124:44773
I20260811 12:47:13.668318 19386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.124:44773 every 8 connection(s)
I20260811 12:47:13.669829 18963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/master-2/data/info.pb
I20260811 12:47:13.675534 19387 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:47:13.679175 19387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd: Bootstrap starting.
I20260811 12:47:13.698714 19387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:47:13.699247 19387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd: Bootstrap complete.
I20260811 12:47:13.700366 19387 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: NON_VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: true } }
I20260811 12:47:13.700745 19387 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88bd75212b2e4308a018df055d5b6ccd, State: Initialized, Role: LEARNER
I20260811 12:47:13.701041 19387 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: NON_VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: true } }
I20260811 12:47:13.702448 19387 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:47:13.702195 19393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: NON_VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: true } } }
I20260811 12:47:13.702733 19393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [sys.catalog]: This master's current role is: LEARNER
I20260811 12:47:13.718845 19411 catalog_manager.cc:1259] Loaded cluster ID: 53e0569ae84d4f13b3a2305a0117f25e
I20260811 12:47:13.718966 19411 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd: loading cluster ID for follower catalog manager: success
I20260811 12:47:13.722504 19411 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd: acquiring CA information for follower catalog manager: success
I20260811 12:47:13.724937 19411 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 12:47:13.955363 19295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:13.955657 19295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:13.955797 19295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:13.963068 19295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:13.963166 19295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:13.963310 19295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:47:13.972854 19295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:42611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.65.keytab
--principal=oryx/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:33335
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=43627
--tserver_master_addrs=127.7.111.126:34613,127.7.111.125:38817,127.7.111.124:44773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:13.974182 19295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:13.975492 19295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:13.988494 19419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:13.988528 19420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:13.988667 19423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:13.989333 19295 server_base.cc:1034] running on GCE node
I20260811 12:47:13.989928 19295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:13.990826 19295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:13.992020 19295 hybrid_clock.cc:648] HybridClock initialized: now 1786452433991968 us; error 42 us; skew 500 ppm
I20260811 12:47:13.992317 19295 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:13 dist-test-slave-f0tn krb5kdc[18024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452433, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:13.995543 19295 init.cc:377] Logged in from keytab as oryx/127.7.111.65@KRBTEST.COM (short username oryx)
I20260811 12:47:13.996902 19295 webserver.cc:460] Webserver started at http://127.7.111.65:43627/ using document root <none> and password file <none>
I20260811 12:47:13.997609 19295 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:13.997756 19295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:14.003103 19295 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 12:47:14.005785 19434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:14.007093 19295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:47:14.007229 19295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "783323bb150f42aeb023d8577443ffc0"
format_stamp: "Formatted at 2026-08-11 12:47:10 on dist-test-slave-f0tn"
server_key: "791dc13123f155ec0d70b67febe5d42d"
server_key_iv: "0bf3f2d4eea2abec044068748a6bc8a6"
server_key_version: "encryptionkey@0"
I20260811 12:47:14.008194 19295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:14.016850 19436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:14.009005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:44773 (local address 127.0.0.1:53018)
0811 12:47:14.009883 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:14.009900 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:14.010025 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:14.010432 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:14.010435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:14.010453 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:47:14.010699 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:14.010709 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.012636 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.012640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:14.014194 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:14.014210 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.016174 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.016181 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:14.016371 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:14.016397 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 12:47:14.016476 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 12:47:14.016578 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"thread_start_us":683,"threads_started":1}
I20260811 12:47:14.017019 19438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:14.009188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53018 (local address 127.7.111.124:44773)
0811 12:47:14.009736 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:14.009742 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:14.009982 (+ 240us) server_negotiation.cc:407] Connection header received
0811 12:47:14.010158 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:14.010163 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:14.010292 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:14.010432 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:47:14.010850 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.012495 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.014352 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.016004 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.016144 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:14.016352 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:14.016510 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 12:47:14.016635 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":267,"threads_started":1}
I20260811 12:47:14.017804 19345 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 12:47:14.018427 19393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8acd3cadf1424c2792ed3fc7e4c5f506. Latest consensus state: current_term: 1 leader_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: NON_VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: true } } }
I20260811 12:47:14.018692 19393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [sys.catalog]: This master's current role is: LEARNER
I20260811 12:47:14.030336 19295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:14.035175 19295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:14.035478 19295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:14.036681 19295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:14.038321 19295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:14.038439 19295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:14.038555 19295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:14.038659 19295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:14.082846 19295 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:33335
I20260811 12:47:14.082966 19616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:33335 every 8 connection(s)
I20260811 12:47:14.084450 19295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:47:14.086611 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 19295
I20260811 12:47:14.086972 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 18388
Aug 11 12:47:14 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.111.66@KRBTEST.COM
Aug 11 12:47:14 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452433, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.124@KRBTEST.COM
Aug 11 12:47:14 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452433, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM
Aug 11 12:47:14 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452433, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.125@KRBTEST.COM
I20260811 12:47:14.184237 19625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:14.088297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:34077 (local address 127.7.111.126:34613)
0811 12:47:14.158771 (+ 70474us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:14.158779 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:14.164159 (+ 5380us) server_negotiation.cc:407] Connection header received
0811 12:47:14.164410 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:14.164415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:14.164498 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:14.164576 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:14.166332 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.167549 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.170483 (+ 2934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.171202 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.171301 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:14.177136 (+ 5835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:14.177142 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:14.177145 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:14.177188 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:14.180523 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:14.181240 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:14.181245 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:14.181249 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:14.181408 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:14.181966 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:14.181971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:14.181974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:14.182324 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:14.182451 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:14.183585 (+ 1134us) server_negotiation.cc:299] Negotiation successful
0811 12:47:14.183852 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70300,"thread_start_us":70585,"threads_started":1}
I20260811 12:47:14.184988 19438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:14.089557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:51791 (local address 127.7.111.124:44773)
0811 12:47:14.089775 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:14.089782 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:14.153762 (+ 63980us) server_negotiation.cc:407] Connection header received
0811 12:47:14.163491 (+ 9729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:14.163497 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:14.163578 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:14.163668 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:14.166246 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.167284 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.170778 (+ 3494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.171454 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.171562 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:14.176789 (+ 5227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:14.176795 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:14.176800 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:14.176845 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:14.181802 (+ 4957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:14.182559 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:14.182565 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:14.182568 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:14.182734 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:14.183229 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:14.183235 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:14.183238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:14.183596 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:14.183725 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:14.184266 (+ 541us) server_negotiation.cc:299] Negotiation successful
0811 12:47:14.184559 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 12:47:14.185411 19629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:14.088808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49706 (local address 127.7.111.66:46081)
0811 12:47:14.152497 (+ 63689us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:14.152510 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:14.160297 (+ 7787us) server_negotiation.cc:407] Connection header received
0811 12:47:14.160372 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:14.160379 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:14.160655 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:14.160803 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:14.163170 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.165299 (+ 2129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.167392 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.168341 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.168464 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:14.175588 (+ 7124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:14.175609 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:14.175630 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:14.175692 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:14.179999 (+ 4307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:14.180953 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:14.180962 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:14.180966 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:14.181149 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:14.181640 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:14.181646 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:14.181649 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:14.182144 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:14.184545 (+ 2401us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:14.184712 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 12:47:14.184979 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63373,"thread_start_us":68617,"threads_started":1}
I20260811 12:47:14.186697 19628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:14.090811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.65:34077)
0811 12:47:14.164063 (+ 73252us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:14.164081 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:14.164258 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:14.164895 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:14.164899 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:14.165669 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:14.166160 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:14.166170 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.167743 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.167747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:14.170312 (+ 2565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:14.170328 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.171697 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.171706 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:14.172126 (+ 420us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:14.173012 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:14.173053 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:14.176893 (+ 3840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:14.180647 (+ 3754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:14.180654 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:14.180669 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:14.181107 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:14.181552 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:14.181556 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:14.181559 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:14.181840 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:14.182546 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:14.182555 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:14.183051 (+ 496us) client_negotiation.cc:769] Sending connection context
0811 12:47:14.184539 (+ 1488us) client_negotiation.cc:240] Negotiation successful
0811 12:47:14.185103 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73124,"mutex_wait_us":252,"spinlock_wait_cycles":2048,"thread_start_us":65941,"threads_started":1}
I20260811 12:47:14.186744 18574 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:14.186746 19630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:14.088176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:44773 (local address 127.7.111.65:51791)
0811 12:47:14.153470 (+ 65294us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:14.153563 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:14.163095 (+ 9532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:14.164051 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:14.164063 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:14.164949 (+ 886us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:14.165968 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:14.165999 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.167472 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.167477 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:14.170594 (+ 3117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:14.170610 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.171928 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.171935 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:14.172173 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:14.173159 (+ 986us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:14.173182 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:14.176284 (+ 3102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:14.181956 (+ 5672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:14.181960 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:14.181963 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:14.182427 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:14.182880 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:14.182884 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:14.182888 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:14.183074 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:14.183811 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:14.183815 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:14.183971 (+ 156us) client_negotiation.cc:769] Sending connection context
0811 12:47:14.185169 (+ 1198us) client_negotiation.cc:240] Negotiation successful
0811 12:47:14.185522 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64815,"mutex_wait_us":159,"spinlock_wait_cycles":896,"thread_start_us":73690,"threads_started":1}
I20260811 12:47:14.188555 19619 heartbeater.cc:347] Connected to a master server at 127.7.111.126:34613
I20260811 12:47:14.188576 19617 heartbeater.cc:347] Connected to a master server at 127.7.111.124:44773
I20260811 12:47:14.188923 19617 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:14.188930 19619 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:14.189973 19619 heartbeater.cc:511] Master 127.7.111.126:34613 requested a full tablet report, sending...
I20260811 12:47:14.190516 19623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:14.087928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:44361 (local address 127.7.111.125:38817)
0811 12:47:14.159386 (+ 71458us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:14.159394 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:14.162500 (+ 3106us) server_negotiation.cc:407] Connection header received
0811 12:47:14.164184 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:14.164190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:14.164292 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:14.164391 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:14.166333 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.167550 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.170483 (+ 2933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.171202 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.171311 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:14.179524 (+ 8213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:14.179531 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:14.179536 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:14.179583 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:14.183294 (+ 3711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:14.184252 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:14.184257 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:14.184260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:14.184688 (+ 428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:14.186451 (+ 1763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:14.186458 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:14.186461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:14.187839 (+ 1378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:14.188279 (+ 440us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:14.189752 (+ 1473us) server_negotiation.cc:299] Negotiation successful
0811 12:47:14.190018 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71300,"thread_start_us":73103,"threads_started":1}
I20260811 12:47:14.191202 19617 heartbeater.cc:511] Master 127.7.111.124:44773 requested a full tablet report, sending...
I20260811 12:47:14.191794 19626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:14.088180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:38817 (local address 127.7.111.65:44361)
0811 12:47:14.162306 (+ 74126us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:14.162326 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:14.163980 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:14.164608 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:14.164612 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:14.165669 (+ 1057us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:14.166150 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:14.166162 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.167741 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.167747 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:14.170312 (+ 2565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:14.170328 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.171696 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.171706 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:14.172103 (+ 397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:14.173012 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:14.173053 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:14.179265 (+ 6212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:14.183460 (+ 4195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:14.183464 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:14.183468 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:14.184102 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:14.184839 (+ 737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:14.184843 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:14.184847 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:14.185026 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:14.188371 (+ 3345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:14.188377 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:14.189177 (+ 800us) client_negotiation.cc:769] Sending connection context
0811 12:47:14.190763 (+ 1586us) client_negotiation.cc:240] Negotiation successful
0811 12:47:14.191201 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73951,"mutex_wait_us":295,"spinlock_wait_cycles":64896,"thread_start_us":75396,"threads_started":1}
I20260811 12:47:14.192826 18089 ts_manager.cc:194] Re-registered known tserver with Master: 783323bb150f42aeb023d8577443ffc0 (127.7.111.65:33335)
I20260811 12:47:14.193171 19330 ts_manager.cc:194] Registered new tserver with Master: 783323bb150f42aeb023d8577443ffc0 (127.7.111.65:33335)
I20260811 12:47:14.193696 18089 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:34077
I20260811 12:47:14.194614 19620 heartbeater.cc:347] Connected to a master server at 127.7.111.125:38817
I20260811 12:47:14.194761 19620 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:14.195480 19620 heartbeater.cc:511] Master 127.7.111.125:38817 requested a full tablet report, sending...
I20260811 12:47:14.197173 19030 ts_manager.cc:194] Registered new tserver with Master: 783323bb150f42aeb023d8577443ffc0 (127.7.111.65:33335)
I20260811 12:47:14.274686 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 18388
I20260811 12:47:14.300683 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:46081
--local_ip_for_outbound_sockets=127.7.111.66
--tserver_master_addrs=127.7.111.126:34613
--webserver_port=41233
--webserver_interface=127.7.111.66
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.66.keytab
--principal=oryx/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:42611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.111.126:34613,127.7.111.125:38817
--tserver_master_addrs=127.7.111.126:34613,127.7.111.125:38817,127.7.111.124:44773 with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:47:14.562884 19435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: NON_VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: true } } }
I20260811 12:47:14.563241 19435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:14.564509 19635 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506: attempting to promote NON_VOTER 88bd75212b2e4308a018df055d5b6ccd to VOTER
I20260811 12:47:14.565075 19635 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: false } }
I20260811 12:47:14.567121 19345 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 1 LEARNER]: Refusing update from remote peer 8acd3cadf1424c2792ed3fc7e4c5f506: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 12:47:14.567117 19043 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 1 FOLLOWER]: Refusing update from remote peer 8acd3cadf1424c2792ed3fc7e4c5f506: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 12:47:14.567754 19635 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 12:47:14.568111 19435 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [LEADER]: Connected to new peer: Peer: permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 12:47:14.572331 19435 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 88bd75212b2e4308a018df055d5b6ccd (127.7.111.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: false } } }
I20260811 12:47:14.577075 19635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: false } } }
I20260811 12:47:14.576659 19345 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 88bd75212b2e4308a018df055d5b6ccd (127.7.111.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: false } } }
I20260811 12:47:14.577361 19635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:14.577160 19435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: false } } }
I20260811 12:47:14.577659 19435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:14.577953 19635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: false } } }
I20260811 12:47:14.578194 19635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:14.578437 19043 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 88bd75212b2e4308a018df055d5b6ccd (127.7.111.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: false } } }
I20260811 12:47:14.581096 19645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: false } } }
I20260811 12:47:14.581507 19645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:47:14.582360 19646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: false } } }
I20260811 12:47:14.582729 19646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [sys.catalog]: This master's current role is: FOLLOWER
W20260811 12:47:14.646490 19632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:14.646761 19632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:14.646896 19632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:14.654094 19632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:14.654220 19632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:14.654399 19632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:47:14.665028 19632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:42611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.66.keytab
--principal=oryx/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:46081
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=41233
--tserver_master_addrs=127.7.111.126:34613,127.7.111.125:38817,127.7.111.124:44773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:14.666409 19632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:14.667861 19632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:14.680994 19657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:14.681063 19655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:14.681110 19653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:14.682077 19632 server_base.cc:1034] running on GCE node
I20260811 12:47:14.682642 19632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:14.683537 19632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:14.684749 19632 hybrid_clock.cc:648] HybridClock initialized: now 1786452434684705 us; error 33 us; skew 500 ppm
I20260811 12:47:14.685047 19632 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:14 dist-test-slave-f0tn krb5kdc[18024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452434, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:14.688174 19632 init.cc:377] Logged in from keytab as oryx/127.7.111.66@KRBTEST.COM (short username oryx)
I20260811 12:47:14.689527 19632 webserver.cc:460] Webserver started at http://127.7.111.66:41233/ using document root <none> and password file <none>
I20260811 12:47:14.690186 19632 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:14.690315 19632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:14.695716 19632 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 12:47:14.698344 19672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:14.699651 19632 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:47:14.699833 19632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "16939728527c4f3287b355862cc6a6f0"
format_stamp: "Formatted at 2026-08-11 12:47:11 on dist-test-slave-f0tn"
server_key: "db0873f4f8df918fb90fb9a2fd55ab54"
server_key_iv: "1ec8adb18ee46e42bfaba198edcb27f0"
server_key_version: "encryptionkey@0"
I20260811 12:47:14.700785 19632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:14.726915 19632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:14.731005 19632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:14.731263 19632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:14.732244 19632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:14.733647 19632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:14.733749 19632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:14.733852 19632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:14.733922 19632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:14.764130 19632 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:46081
I20260811 12:47:14.764195 19848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:46081 every 8 connection(s)
I20260811 12:47:14.765530 19632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:47:14.766273 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 19632
I20260811 12:47:14.766592 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 18591
Aug 11 12:47:14 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.111.67@KRBTEST.COM
I20260811 12:47:14.817584 19856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:14.767997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41228 (local address 127.7.111.67:39893)
0811 12:47:14.790607 (+ 22610us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:14.790618 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:14.790681 (+ 63us) server_negotiation.cc:407] Connection header received
0811 12:47:14.790773 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:14.790783 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:14.791079 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:14.791248 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:14.794372 (+ 3124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.796145 (+ 1773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.798292 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.799240 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.799368 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:14.807808 (+ 8440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:14.807832 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:14.807855 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:14.807926 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:14.812700 (+ 4774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:14.813721 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:14.813731 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:14.813735 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:14.813922 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:14.814485 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:14.814491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:14.814506 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:14.814969 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:14.815298 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:14.816773 (+ 1475us) server_negotiation.cc:299] Negotiation successful
0811 12:47:14.817064 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22433,"thread_start_us":22289,"threads_started":1}
I20260811 12:47:14.819653 18774 generic_service.cc:196] Checking for leaks (request via RPC)
Aug 11 12:47:14 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452434, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.125@KRBTEST.COM
Aug 11 12:47:14 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452434, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.124@KRBTEST.COM
Aug 11 12:47:14 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452434, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM
I20260811 12:47:14.833694 19859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:14.768382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:60799 (local address 127.7.111.126:34613)
0811 12:47:14.810913 (+ 42531us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:14.810920 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:14.810943 (+ 23us) server_negotiation.cc:407] Connection header received
0811 12:47:14.811029 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:14.811054 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:14.811140 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:14.811253 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:14.814322 (+ 3069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.815499 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.818543 (+ 3044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.819288 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.819440 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:14.825948 (+ 6508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:14.825955 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:14.825960 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:14.826039 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:14.830126 (+ 4087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:14.830893 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:14.830898 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:14.830901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:14.831057 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:14.831493 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:14.831497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:14.831500 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:14.831836 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:14.831965 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:14.833084 (+ 1119us) server_negotiation.cc:299] Negotiation successful
0811 12:47:14.833310 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42368,"thread_start_us":42263,"threads_started":1}
I20260811 12:47:14.833936 19858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:14.768376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:59021 (local address 127.7.111.124:44773)
0811 12:47:14.813923 (+ 45547us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:14.813929 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:14.813947 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:47:14.814008 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:14.814012 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:14.814092 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:14.814200 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:14.815729 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.816857 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.818851 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.819577 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.819680 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:14.825163 (+ 5483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:14.825169 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:14.825173 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:14.825211 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:14.829037 (+ 3826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:14.830572 (+ 1535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:14.830580 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:14.830584 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:14.830836 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:14.831471 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:14.831478 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:14.831483 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:14.831885 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:14.832063 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:14.833118 (+ 1055us) server_negotiation.cc:299] Negotiation successful
0811 12:47:14.833411 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45424,"thread_start_us":45288,"threads_started":1}
I20260811 12:47:14.834582 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:14.775879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:39515 (local address 127.7.111.125:38817)
0811 12:47:14.809508 (+ 33629us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:14.809514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:14.813084 (+ 3570us) server_negotiation.cc:407] Connection header received
0811 12:47:14.813290 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:14.813296 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:14.813423 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:14.813594 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:14.815382 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.816543 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.818672 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.819518 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.819622 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:14.824964 (+ 5342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:14.824971 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:14.824975 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:14.825026 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:14.829087 (+ 4061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:14.830515 (+ 1428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:14.830520 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:14.830524 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:14.830710 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:14.831249 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:14.831254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:14.831257 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:14.831659 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:14.831814 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:14.834137 (+ 2323us) server_negotiation.cc:299] Negotiation successful
0811 12:47:14.834340 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33470,"thread_start_us":33644,"threads_started":1}
I20260811 12:47:14.835966 19868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:14.769017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.66:60799)
0811 12:47:14.805534 (+ 36517us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:14.805609 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:14.808548 (+ 2939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:14.811678 (+ 3130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:14.811698 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:14.812826 (+ 1128us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:14.814041 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:14.814074 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.815738 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.815745 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:14.818260 (+ 2515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:14.818284 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.819760 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.819769 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:14.820155 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:14.821021 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:14.821059 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:14.825623 (+ 4564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:14.830290 (+ 4667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:14.830295 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:14.830298 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:14.830764 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:14.831189 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:14.831194 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:14.831197 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:14.831363 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:14.832018 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:14.832022 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:14.832373 (+ 351us) client_negotiation.cc:769] Sending connection context
0811 12:47:14.832702 (+ 329us) client_negotiation.cc:240] Negotiation successful
0811 12:47:14.835047 (+ 2345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36016,"mutex_wait_us":1921,"spinlock_wait_cycles":22912,"thread_start_us":43239,"threads_started":1}
I20260811 12:47:14.836025 19862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:14.769588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:44773 (local address 127.7.111.66:59021)
0811 12:47:14.811947 (+ 42359us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:14.811966 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:14.812105 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:14.814414 (+ 2309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:14.814418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:14.815252 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:14.815558 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:14.815571 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.817087 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.817093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:14.818698 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:14.818710 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.819992 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.819999 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:14.820284 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:14.821415 (+ 1131us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:14.821438 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:14.824819 (+ 3381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:14.829250 (+ 4431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:14.829262 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:14.829282 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:14.830383 (+ 1101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:14.831026 (+ 643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:14.831033 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:14.831037 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:14.831303 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:14.832138 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:14.832144 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:14.832372 (+ 228us) client_negotiation.cc:769] Sending connection context
0811 12:47:14.832705 (+ 333us) client_negotiation.cc:240] Negotiation successful
0811 12:47:14.834558 (+ 1853us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42186,"mutex_wait_us":1429,"spinlock_wait_cycles":61440,"thread_start_us":42011,"threads_started":1}
I20260811 12:47:14.837024 19860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:14.775409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:38817 (local address 127.7.111.66:39515)
0811 12:47:14.812876 (+ 37467us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:14.812896 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:14.813123 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:14.813806 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:14.813810 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:14.814681 (+ 871us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:14.815190 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:14.815205 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.816727 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.816732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:14.818509 (+ 1777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:14.818525 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:14.819977 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:14.819984 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:14.820260 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:14.821149 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:14.821173 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:14.824704 (+ 3531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:14.829249 (+ 4545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:14.829262 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:14.829281 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:14.830318 (+ 1037us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:14.830846 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:14.830850 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:14.830853 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:14.831112 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:14.831893 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:14.831902 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:14.832604 (+ 702us) client_negotiation.cc:769] Sending connection context
0811 12:47:14.832738 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 12:47:14.833260 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37290,"mutex_wait_us":508,"thread_start_us":34209,"threads_started":1}
I20260811 12:47:14.837966 19853 heartbeater.cc:347] Connected to a master server at 127.7.111.125:38817
I20260811 12:47:14.838406 19853 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:14.838433 19852 heartbeater.cc:347] Connected to a master server at 127.7.111.126:34613
I20260811 12:47:14.838577 19852 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:14.838423 19849 heartbeater.cc:347] Connected to a master server at 127.7.111.124:44773
I20260811 12:47:14.839859 19849 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:14.839838 19853 heartbeater.cc:511] Master 127.7.111.125:38817 requested a full tablet report, sending...
I20260811 12:47:14.840103 19852 heartbeater.cc:511] Master 127.7.111.126:34613 requested a full tablet report, sending...
I20260811 12:47:14.840621 19849 heartbeater.cc:511] Master 127.7.111.124:44773 requested a full tablet report, sending...
I20260811 12:47:14.841912 18089 ts_manager.cc:194] Re-registered known tserver with Master: 16939728527c4f3287b355862cc6a6f0 (127.7.111.66:46081)
I20260811 12:47:14.841976 19330 ts_manager.cc:194] Registered new tserver with Master: 16939728527c4f3287b355862cc6a6f0 (127.7.111.66:46081)
I20260811 12:47:14.842012 19030 ts_manager.cc:194] Registered new tserver with Master: 16939728527c4f3287b355862cc6a6f0 (127.7.111.66:46081)
I20260811 12:47:14.843011 18089 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:60799
I20260811 12:47:14.923254 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 18591
I20260811 12:47:14.951460 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:39893
--local_ip_for_outbound_sockets=127.7.111.67
--tserver_master_addrs=127.7.111.126:34613
--webserver_port=43635
--webserver_interface=127.7.111.67
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.67.keytab
--principal=oryx/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:42611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.111.126:34613,127.7.111.125:38817
--tserver_master_addrs=127.7.111.126:34613,127.7.111.125:38817,127.7.111.124:44773 with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:47:15.196570 19619 heartbeater.cc:503] Master 127.7.111.126:34613 was elected leader, sending a full tablet report...
W20260811 12:47:15.289541 19872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:15.289806 19872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:15.289935 19872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:15.297113 19872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:15.297236 19872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:15.297422 19872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:47:15.307246 19872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:42611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.67.keytab
--principal=oryx/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:39893
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=43635
--tserver_master_addrs=127.7.111.126:34613,127.7.111.125:38817,127.7.111.124:44773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:15.308604 19872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:15.310019 19872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:15.323314 19882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:15.323242 19886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:15.323242 19883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:15.324139 19872 server_base.cc:1034] running on GCE node
I20260811 12:47:15.324714 19872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:15.325688 19872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:15.326884 19872 hybrid_clock.cc:648] HybridClock initialized: now 1786452435326827 us; error 50 us; skew 500 ppm
I20260811 12:47:15.327199 19872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:15 dist-test-slave-f0tn krb5kdc[18024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452435, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:15.330395 19872 init.cc:377] Logged in from keytab as oryx/127.7.111.67@KRBTEST.COM (short username oryx)
I20260811 12:47:15.331980 19872 webserver.cc:460] Webserver started at http://127.7.111.67:43635/ using document root <none> and password file <none>
I20260811 12:47:15.332712 19872 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:15.332849 19872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:15.338610 19872 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 12:47:15.341337 19900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:15.342703 19872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:47:15.342854 19872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "9ae448d0f0be4eb3ac28cf4ff915a958"
format_stamp: "Formatted at 2026-08-11 12:47:11 on dist-test-slave-f0tn"
server_key: "e3d26f5ead72e95831c5dfb5a638232a"
server_key_iv: "78b54127d3b263e8eceab4dcbedeee93"
server_key_version: "encryptionkey@0"
I20260811 12:47:15.343822 19872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:15.360812 19872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:15.364641 19872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:15.364883 19872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:15.365865 19872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:15.367206 19872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:15.367301 19872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:15.367409 19872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:15.367484 19872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:15.399888 19872 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:39893
I20260811 12:47:15.399941 20093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:39893 every 8 connection(s)
I20260811 12:47:15.401453 19872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:47:15.407454 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 19872
Aug 11 12:47:15 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452435, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM
Aug 11 12:47:15 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452435, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.124@KRBTEST.COM
I20260811 12:47:15.468600 20100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:15.404450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:54837 (local address 127.7.111.124:44773)
0811 12:47:15.415644 (+ 11194us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:15.415650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:15.422058 (+ 6408us) server_negotiation.cc:407] Connection header received
0811 12:47:15.422430 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:15.422436 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:15.422538 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:15.422628 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:15.426818 (+ 4190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.428011 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.442443 (+ 14432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.443125 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.443224 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:15.460302 (+ 17078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:15.460310 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:15.460315 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:15.460356 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:15.464379 (+ 4023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:15.465473 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:15.465479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:15.465482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:15.465637 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:15.466921 (+ 1284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:15.466926 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:15.466930 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:15.467288 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:15.467428 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:15.468012 (+ 584us) server_negotiation.cc:299] Negotiation successful
0811 12:47:15.468245 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11046,"spinlock_wait_cycles":14208,"thread_start_us":10932,"threads_started":1}
I20260811 12:47:15.471194 20099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:15.404333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:46111 (local address 127.7.111.126:34613)
0811 12:47:15.417575 (+ 13242us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:15.417582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:15.418416 (+ 834us) server_negotiation.cc:407] Connection header received
0811 12:47:15.425488 (+ 7072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:15.425495 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:15.425584 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:15.425666 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:15.430654 (+ 4988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.431816 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.434345 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.435053 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.435147 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:15.452145 (+ 16998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:15.452154 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:15.452159 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:15.452203 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:15.456094 (+ 3891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:15.458034 (+ 1940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:15.458040 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:15.458044 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:15.458205 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:15.459494 (+ 1289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:15.459499 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:15.459502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:15.459855 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:15.459983 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:15.470586 (+ 10603us) server_negotiation.cc:299] Negotiation successful
0811 12:47:15.470837 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13021,"thread_start_us":13042,"threads_started":1}
I20260811 12:47:15.472295 20101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:15.405676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:44773 (local address 127.7.111.67:54837)
0811 12:47:15.421937 (+ 16261us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:15.421964 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:15.422229 (+ 265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:15.423238 (+ 1009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:15.423251 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:15.424298 (+ 1047us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:15.425221 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:15.425245 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.428230 (+ 2985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.428235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:15.430464 (+ 2229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:15.430480 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.443593 (+ 13113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.443600 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:15.443784 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:15.446846 (+ 3062us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:15.446868 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:15.460108 (+ 13240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:15.464594 (+ 4486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:15.464599 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:15.464603 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:15.465092 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:15.465818 (+ 726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:15.465823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:15.465827 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:15.465998 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:15.467512 (+ 1514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:15.467516 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:15.467675 (+ 159us) client_negotiation.cc:769] Sending connection context
0811 12:47:15.468925 (+ 1250us) client_negotiation.cc:240] Negotiation successful
0811 12:47:15.469496 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16135,"thread_start_us":15957,"threads_started":1}
I20260811 12:47:15.473029 20102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:15.405144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.67:46111)
0811 12:47:15.418227 (+ 13083us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:15.418285 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:15.421177 (+ 2892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:15.425892 (+ 4715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:15.425896 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:15.426341 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:15.426623 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:15.426634 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.432730 (+ 6096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.432734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:15.434177 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:15.434192 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.435554 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.435561 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:15.441667 (+ 6106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:15.445042 (+ 3375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:15.445077 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:15.451426 (+ 6349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:15.456309 (+ 4883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:15.456320 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:15.456340 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:15.457445 (+ 1105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:15.458363 (+ 918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:15.458368 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:15.458372 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:15.458619 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:15.460078 (+ 1459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:15.460086 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:15.466524 (+ 6438us) client_negotiation.cc:769] Sending connection context
0811 12:47:15.470452 (+ 3928us) client_negotiation.cc:240] Negotiation successful
0811 12:47:15.470773 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12769,"thread_start_us":12521,"threads_started":1}
I20260811 12:47:15.475549 20095 heartbeater.cc:347] Connected to a master server at 127.7.111.126:34613
I20260811 12:47:15.475909 20095 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:15.476961 20095 heartbeater.cc:511] Master 127.7.111.126:34613 requested a full tablet report, sending...
I20260811 12:47:15.477448 20094 heartbeater.cc:347] Connected to a master server at 127.7.111.124:44773
I20260811 12:47:15.477586 20094 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:15.478605 20094 heartbeater.cc:511] Master 127.7.111.124:44773 requested a full tablet report, sending...
I20260811 12:47:15.478986 20116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:15.431046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40342 (local address 127.7.111.124:44773)
0811 12:47:15.455214 (+ 24168us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:15.455222 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:15.455246 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:47:15.455314 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:15.455321 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:15.455503 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:15.455629 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:15.459136 (+ 3507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.460700 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.463277 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.464114 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.464255 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:15.470764 (+ 6509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:15.470771 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:15.470776 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:15.470837 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:15.474475 (+ 3638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:15.475567 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:15.475574 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:15.475578 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:15.475757 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:15.476327 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:15.476333 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:15.476336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:15.476727 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:15.477015 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:15.478274 (+ 1259us) server_negotiation.cc:299] Negotiation successful
0811 12:47:15.478512 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23997,"thread_start_us":29778,"threads_started":1}
I20260811 12:47:15.480158 19330 ts_manager.cc:194] Registered new tserver with Master: 9ae448d0f0be4eb3ac28cf4ff915a958 (127.7.111.67:39893)
I20260811 12:47:15.481029 18089 ts_manager.cc:194] Re-registered known tserver with Master: 9ae448d0f0be4eb3ac28cf4ff915a958 (127.7.111.67:39893)
I20260811 12:47:15.482167 18089 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:46111
I20260811 12:47:15.487695 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:15.430251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35584 (local address 127.7.111.125:38817)
0811 12:47:15.465100 (+ 34849us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:15.465106 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:15.465124 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:47:15.465185 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:15.465191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:15.465284 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:15.468760 (+ 3476us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:15.468928 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.470030 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.471863 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.472469 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.474086 (+ 1617us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:15.478326 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:15.478332 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:15.478337 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:15.478375 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:15.483872 (+ 5497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:15.484749 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:15.484755 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:15.484759 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:15.485085 (+ 326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:15.485626 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:15.485633 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:15.485638 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:15.486093 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:15.486381 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:15.487052 (+ 671us) server_negotiation.cc:299] Negotiation successful
0811 12:47:15.487286 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34709,"thread_start_us":32294,"threads_started":1}
Aug 11 12:47:15 dist-test-slave-f0tn krb5kdc[18024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452435, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.125@KRBTEST.COM
I20260811 12:47:15.490687 20117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:15.435897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56754 (local address 127.7.111.126:34613)
0811 12:47:15.459322 (+ 23425us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:15.459328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:15.469859 (+ 10531us) server_negotiation.cc:407] Connection header received
0811 12:47:15.469927 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:15.469932 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:15.470062 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:15.470176 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:15.474610 (+ 4434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.476023 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.478227 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.479642 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.479780 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:15.483575 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:15.483582 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:15.483588 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:15.483633 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:15.487536 (+ 3903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:15.488606 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:15.488610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:15.488613 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:15.488817 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:15.489251 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:15.489255 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:15.489258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:15.489571 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:15.489793 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:15.490105 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 12:47:15.490349 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23275,"thread_start_us":23433,"threads_started":1}
I20260811 12:47:15.496098 20109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:15.423337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:38817 (local address 127.7.111.67:52169)
0811 12:47:15.466056 (+ 42719us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:15.466086 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:15.466267 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:15.472626 (+ 6359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:15.472631 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:15.479766 (+ 7135us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:15.480009 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:15.480022 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.483492 (+ 3470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.483498 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:15.485421 (+ 1923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:15.485435 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.486291 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.486297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:15.486640 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:15.487214 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:15.487237 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:15.489886 (+ 2649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:15.493288 (+ 3402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:15.493294 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:15.493297 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:15.493838 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:15.494223 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:15.494229 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:15.494233 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:15.494379 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:15.494986 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:15.494991 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:15.495173 (+ 182us) client_negotiation.cc:769] Sending connection context
0811 12:47:15.495307 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 12:47:15.495621 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42551,"mutex_wait_us":30,"thread_start_us":46078,"threads_started":1}
I20260811 12:47:15.496098 20114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:15.423817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:52169 (local address 127.7.111.125:38817)
0811 12:47:15.462521 (+ 38704us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:15.462528 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:15.468368 (+ 5840us) server_negotiation.cc:407] Connection header received
0811 12:47:15.468429 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:15.468435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:15.468505 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:15.468604 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:15.481602 (+ 12998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.483333 (+ 1731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.485554 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.486099 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.486221 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:15.490122 (+ 3901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:15.490128 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:15.490132 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:15.490171 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:15.493153 (+ 2982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:15.493971 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:15.493976 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:15.493978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:15.494108 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:15.494504 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:15.494508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:15.494511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:15.494857 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:15.495049 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:15.495473 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 12:47:15.495687 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38571,"thread_start_us":41443,"threads_started":1}
I20260811 12:47:15.496917 20097 heartbeater.cc:347] Connected to a master server at 127.7.111.125:38817
I20260811 12:47:15.497062 20097 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:15.497521 20097 heartbeater.cc:511] Master 127.7.111.125:38817 requested a full tablet report, sending...
I20260811 12:47:15.498665 19030 ts_manager.cc:194] Registered new tserver with Master: 9ae448d0f0be4eb3ac28cf4ff915a958 (127.7.111.67:39893)
I20260811 12:47:15.494504 18089 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:47:15.502419 18089 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:47:15.532135 20140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:15.524665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45614 (local address 127.7.111.65:33335)
0811 12:47:15.525482 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:15.525491 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:15.526068 (+ 577us) server_negotiation.cc:407] Connection header received
0811 12:47:15.526217 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:15.526223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:15.526425 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:15.526590 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:47:15.527205 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.528154 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.530055 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.531138 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.531253 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:15.531488 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:15.531715 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 12:47:15.531876 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":600,"threads_started":1}
I20260811 12:47:15.532204 20141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:15.524006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:33335 (local address 127.0.0.1:45614)
0811 12:47:15.525955 (+ 1949us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:15.525969 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:15.526082 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:15.526591 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:15.526595 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:15.526620 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:47:15.527023 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:15.527034 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.528317 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.528323 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:15.529915 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:15.529927 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.531337 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.531344 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:15.531544 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:15.531572 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 12:47:15.531651 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 12:47:15.531776 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1835,"thread_start_us":819,"threads_started":1}
I20260811 12:47:15.535751 19517 tablet_service.cc:1467] Processing CreateTablet for tablet e188973ab73744c782f70858223e2ee4 (DEFAULT_TABLE table=test-table [id=434d576bdee3498f9804c9a8460ae783]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:47:15.537060 19517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e188973ab73744c782f70858223e2ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:15.549525 20141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:15.525151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:39893 (local address 127.0.0.1:41238)
0811 12:47:15.537589 (+ 12438us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:15.537607 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:15.537730 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:15.538927 (+ 1197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:15.538932 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:15.538957 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:47:15.539119 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:15.539129 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.542868 (+ 3739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.542873 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:15.546402 (+ 3529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:15.546418 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.548604 (+ 2186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.548613 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:15.548797 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:15.548875 (+ 78us) client_negotiation.cc:769] Sending connection context
0811 12:47:15.548972 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 12:47:15.549111 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12352,"mutex_wait_us":50,"thread_start_us":601,"threads_started":1}
I20260811 12:47:15.549571 20142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:15.525444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41238 (local address 127.7.111.67:39893)
0811 12:47:15.537281 (+ 11837us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:15.537290 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:15.538362 (+ 1072us) server_negotiation.cc:407] Connection header received
0811 12:47:15.538483 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:15.538490 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:15.538724 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:15.538925 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:47:15.540283 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.542715 (+ 2432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.546608 (+ 3893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.548302 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.548429 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:15.548806 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:15.549039 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 12:47:15.549213 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11640,"thread_start_us":11851,"threads_started":1}
I20260811 12:47:15.551381 20133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:15.523675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:46081 (local address 127.0.0.1:33820)
0811 12:47:15.524828 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:15.524855 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:15.524985 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:15.544086 (+ 19101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:15.544091 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:15.544117 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:47:15.544876 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:15.544889 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.546326 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.546332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:15.548616 (+ 2284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:15.548632 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.550438 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.550445 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:15.550619 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:15.550651 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:47:15.550738 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 12:47:15.550863 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1023,"mutex_wait_us":208,"thread_start_us":19508,"threads_started":1}
I20260811 12:47:15.552320 20144 tablet_bootstrap.cc:492] T e188973ab73744c782f70858223e2ee4 P 783323bb150f42aeb023d8577443ffc0: Bootstrap starting.
I20260811 12:47:15.553300 20138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:15.524201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33820 (local address 127.7.111.66:46081)
0811 12:47:15.543586 (+ 19385us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:15.543594 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:15.543621 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:47:15.543725 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:15.543732 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:15.543917 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:15.544095 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:47:15.545038 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.546161 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.548880 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.550245 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.550355 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:15.550622 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:15.550793 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 12:47:15.553073 (+ 2280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19225,"thread_start_us":19298,"threads_started":1}
I20260811 12:47:15.553995 19985 tablet_service.cc:1467] Processing CreateTablet for tablet e188973ab73744c782f70858223e2ee4 (DEFAULT_TABLE table=test-table [id=434d576bdee3498f9804c9a8460ae783]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:47:15.555534 19985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e188973ab73744c782f70858223e2ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:15.556416 19740 tablet_service.cc:1467] Processing CreateTablet for tablet e188973ab73744c782f70858223e2ee4 (DEFAULT_TABLE table=test-table [id=434d576bdee3498f9804c9a8460ae783]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:47:15.557080 20144 tablet_bootstrap.cc:654] T e188973ab73744c782f70858223e2ee4 P 783323bb150f42aeb023d8577443ffc0: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:15.557813 19740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e188973ab73744c782f70858223e2ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:15.558892 20144 log.cc:824] T e188973ab73744c782f70858223e2ee4 P 783323bb150f42aeb023d8577443ffc0: Log is configured to *not* fsync() on all Append() calls
I20260811 12:47:15.562448 20144 tablet_bootstrap.cc:492] T e188973ab73744c782f70858223e2ee4 P 783323bb150f42aeb023d8577443ffc0: No bootstrap required, opened a new log
I20260811 12:47:15.562729 20144 ts_tablet_manager.cc:1397] T e188973ab73744c782f70858223e2ee4 P 783323bb150f42aeb023d8577443ffc0: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 12:47:15.571687 20148 tablet_bootstrap.cc:492] T e188973ab73744c782f70858223e2ee4 P 16939728527c4f3287b355862cc6a6f0: Bootstrap starting.
I20260811 12:47:15.572630 20149 tablet_bootstrap.cc:492] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958: Bootstrap starting.
I20260811 12:47:15.574465 20144 raft_consensus.cc:348] T e188973ab73744c782f70858223e2ee4 P 783323bb150f42aeb023d8577443ffc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16939728527c4f3287b355862cc6a6f0" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46081 } } peers { permanent_uuid: "783323bb150f42aeb023d8577443ffc0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33335 } } peers { permanent_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 39893 } }
I20260811 12:47:15.574882 20144 raft_consensus.cc:374] T e188973ab73744c782f70858223e2ee4 P 783323bb150f42aeb023d8577443ffc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:15.574985 20144 raft_consensus.cc:729] T e188973ab73744c782f70858223e2ee4 P 783323bb150f42aeb023d8577443ffc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 783323bb150f42aeb023d8577443ffc0, State: Initialized, Role: FOLLOWER
I20260811 12:47:15.575567 20144 consensus_queue.cc:260] T e188973ab73744c782f70858223e2ee4 P 783323bb150f42aeb023d8577443ffc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16939728527c4f3287b355862cc6a6f0" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46081 } } peers { permanent_uuid: "783323bb150f42aeb023d8577443ffc0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33335 } } peers { permanent_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 39893 } }
I20260811 12:47:15.576421 20148 tablet_bootstrap.cc:654] T e188973ab73744c782f70858223e2ee4 P 16939728527c4f3287b355862cc6a6f0: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:15.576476 20149 tablet_bootstrap.cc:654] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:15.577883 20144 ts_tablet_manager.cc:1428] T e188973ab73744c782f70858223e2ee4 P 783323bb150f42aeb023d8577443ffc0: Time spent starting tablet: real 0.015s user 0.008s sys 0.008s
I20260811 12:47:15.578068 20148 log.cc:824] T e188973ab73744c782f70858223e2ee4 P 16939728527c4f3287b355862cc6a6f0: Log is configured to *not* fsync() on all Append() calls
I20260811 12:47:15.578971 20149 log.cc:824] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958: Log is configured to *not* fsync() on all Append() calls
I20260811 12:47:15.585644 20148 tablet_bootstrap.cc:492] T e188973ab73744c782f70858223e2ee4 P 16939728527c4f3287b355862cc6a6f0: No bootstrap required, opened a new log
I20260811 12:47:15.585700 20149 tablet_bootstrap.cc:492] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958: No bootstrap required, opened a new log
I20260811 12:47:15.585987 20149 ts_tablet_manager.cc:1397] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.005s
I20260811 12:47:15.586037 20148 ts_tablet_manager.cc:1397] T e188973ab73744c782f70858223e2ee4 P 16939728527c4f3287b355862cc6a6f0: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.000s
W20260811 12:47:15.594805 19622 tablet.cc:2369] T e188973ab73744c782f70858223e2ee4 P 783323bb150f42aeb023d8577443ffc0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:47:15.597167 20149 raft_consensus.cc:348] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16939728527c4f3287b355862cc6a6f0" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46081 } } peers { permanent_uuid: "783323bb150f42aeb023d8577443ffc0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33335 } } peers { permanent_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 39893 } }
I20260811 12:47:15.597700 20149 raft_consensus.cc:374] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:15.597810 20149 raft_consensus.cc:729] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ae448d0f0be4eb3ac28cf4ff915a958, State: Initialized, Role: FOLLOWER
I20260811 12:47:15.597647 20148 raft_consensus.cc:348] T e188973ab73744c782f70858223e2ee4 P 16939728527c4f3287b355862cc6a6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16939728527c4f3287b355862cc6a6f0" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46081 } } peers { permanent_uuid: "783323bb150f42aeb023d8577443ffc0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33335 } } peers { permanent_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 39893 } }
I20260811 12:47:15.598165 20148 raft_consensus.cc:374] T e188973ab73744c782f70858223e2ee4 P 16939728527c4f3287b355862cc6a6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:15.598340 20148 raft_consensus.cc:729] T e188973ab73744c782f70858223e2ee4 P 16939728527c4f3287b355862cc6a6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16939728527c4f3287b355862cc6a6f0, State: Initialized, Role: FOLLOWER
I20260811 12:47:15.598631 20149 consensus_queue.cc:260] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16939728527c4f3287b355862cc6a6f0" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46081 } } peers { permanent_uuid: "783323bb150f42aeb023d8577443ffc0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33335 } } peers { permanent_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 39893 } }
I20260811 12:47:15.599010 20148 consensus_queue.cc:260] T e188973ab73744c782f70858223e2ee4 P 16939728527c4f3287b355862cc6a6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16939728527c4f3287b355862cc6a6f0" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46081 } } peers { permanent_uuid: "783323bb150f42aeb023d8577443ffc0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33335 } } peers { permanent_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 39893 } }
I20260811 12:47:15.600653 20149 ts_tablet_manager.cc:1428] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958: Time spent starting tablet: real 0.015s user 0.012s sys 0.002s
I20260811 12:47:15.600952 20095 heartbeater.cc:503] Master 127.7.111.126:34613 was elected leader, sending a full tablet report...
I20260811 12:47:15.601413 19852 heartbeater.cc:503] Master 127.7.111.126:34613 was elected leader, sending a full tablet report...
I20260811 12:47:15.602102 20148 ts_tablet_manager.cc:1428] T e188973ab73744c782f70858223e2ee4 P 16939728527c4f3287b355862cc6a6f0: Time spent starting tablet: real 0.016s user 0.007s sys 0.008s
W20260811 12:47:15.679543 20107 tablet.cc:2369] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:47:15.699679 20155 raft_consensus.cc:482] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:47:15.700024 20155 raft_consensus.cc:504] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16939728527c4f3287b355862cc6a6f0" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46081 } } peers { permanent_uuid: "783323bb150f42aeb023d8577443ffc0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33335 } } peers { permanent_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 39893 } }
I20260811 12:47:15.702082 20155 leader_election.cc:288] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16939728527c4f3287b355862cc6a6f0 (127.7.111.66:46081), 783323bb150f42aeb023d8577443ffc0 (127.7.111.65:33335)
I20260811 12:47:15.708215 20109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:15.702291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:46081 (local address 127.7.111.67:55957)
0811 12:47:15.702468 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:15.702505 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:15.702632 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:15.703182 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:15.703187 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:15.703214 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:47:15.703426 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:15.703436 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.704645 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.704648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:15.706254 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:15.706266 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.707422 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.707428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:15.707670 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:15.707710 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 12:47:15.707791 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 12:47:15.707921 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":26}
I20260811 12:47:15.708251 20138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:15.702690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:55957 (local address 127.7.111.66:46081)
0811 12:47:15.702884 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:15.702888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:15.702902 (+ 14us) server_negotiation.cc:407] Connection header received
0811 12:47:15.702968 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:15.702972 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:15.703044 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:15.703124 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:47:15.703573 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.704485 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.706397 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.707220 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.707371 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:15.707430 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:15.707859 (+ 429us) server_negotiation.cc:299] Negotiation successful
0811 12:47:15.707994 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:47:15.708276 20101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:15.702591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:33335 (local address 127.7.111.67:37769)
0811 12:47:15.702912 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:15.702929 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:15.703038 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:15.703358 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:15.703363 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:15.703388 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:47:15.703594 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:15.703602 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.704661 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.704666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:15.706298 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:15.706311 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.707551 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.707557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:15.707741 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:15.707774 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:47:15.707871 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 12:47:15.707998 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":71}
I20260811 12:47:15.708734 20140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:15.702665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:37769 (local address 127.7.111.65:33335)
0811 12:47:15.702876 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:15.702882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:15.702996 (+ 114us) server_negotiation.cc:407] Connection header received
0811 12:47:15.703161 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:15.703166 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:15.703243 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:15.703319 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:47:15.703722 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.704544 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.706430 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.707403 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.707552 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:15.707624 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:15.707896 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 12:47:15.708021 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:47:15.709698 19774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188973ab73744c782f70858223e2ee4" candidate_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16939728527c4f3287b355862cc6a6f0" is_pre_election: true
I20260811 12:47:15.709815 19545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188973ab73744c782f70858223e2ee4" candidate_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "783323bb150f42aeb023d8577443ffc0" is_pre_election: true
I20260811 12:47:15.710318 19545 raft_consensus.cc:2454] T e188973ab73744c782f70858223e2ee4 P 783323bb150f42aeb023d8577443ffc0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ae448d0f0be4eb3ac28cf4ff915a958 in term 0.
I20260811 12:47:15.710371 19774 raft_consensus.cc:2454] T e188973ab73744c782f70858223e2ee4 P 16939728527c4f3287b355862cc6a6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ae448d0f0be4eb3ac28cf4ff915a958 in term 0.
I20260811 12:47:15.711211 19903 leader_election.cc:302] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 16939728527c4f3287b355862cc6a6f0, 783323bb150f42aeb023d8577443ffc0, 9ae448d0f0be4eb3ac28cf4ff915a958; no voters:
I20260811 12:47:15.711737 20155 raft_consensus.cc:2781] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:47:15.711920 20155 raft_consensus.cc:482] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:47:15.712087 20155 raft_consensus.cc:3037] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:15.714704 20155 raft_consensus.cc:504] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16939728527c4f3287b355862cc6a6f0" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46081 } } peers { permanent_uuid: "783323bb150f42aeb023d8577443ffc0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33335 } } peers { permanent_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 39893 } }
I20260811 12:47:15.715303 20155 leader_election.cc:288] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958 [CANDIDATE]: Term 1 election: Requested vote from peers 16939728527c4f3287b355862cc6a6f0 (127.7.111.66:46081), 783323bb150f42aeb023d8577443ffc0 (127.7.111.65:33335)
I20260811 12:47:15.715794 19545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188973ab73744c782f70858223e2ee4" candidate_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "783323bb150f42aeb023d8577443ffc0"
I20260811 12:47:15.715878 19774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e188973ab73744c782f70858223e2ee4" candidate_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16939728527c4f3287b355862cc6a6f0"
I20260811 12:47:15.716013 19545 raft_consensus.cc:3037] T e188973ab73744c782f70858223e2ee4 P 783323bb150f42aeb023d8577443ffc0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:15.716080 19774 raft_consensus.cc:3037] T e188973ab73744c782f70858223e2ee4 P 16939728527c4f3287b355862cc6a6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:15.719274 19545 raft_consensus.cc:2454] T e188973ab73744c782f70858223e2ee4 P 783323bb150f42aeb023d8577443ffc0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ae448d0f0be4eb3ac28cf4ff915a958 in term 1.
I20260811 12:47:15.719370 19774 raft_consensus.cc:2454] T e188973ab73744c782f70858223e2ee4 P 16939728527c4f3287b355862cc6a6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ae448d0f0be4eb3ac28cf4ff915a958 in term 1.
I20260811 12:47:15.719756 19904 leader_election.cc:302] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 783323bb150f42aeb023d8577443ffc0, 9ae448d0f0be4eb3ac28cf4ff915a958; no voters:
I20260811 12:47:15.720053 20155 raft_consensus.cc:2781] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:47:15.720515 20155 raft_consensus.cc:686] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 1 LEADER]: Becoming Leader. State: Replica: 9ae448d0f0be4eb3ac28cf4ff915a958, State: Running, Role: LEADER
I20260811 12:47:15.721053 20155 consensus_queue.cc:237] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16939728527c4f3287b355862cc6a6f0" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46081 } } peers { permanent_uuid: "783323bb150f42aeb023d8577443ffc0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33335 } } peers { permanent_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 39893 } }
I20260811 12:47:15.726438 18088 catalog_manager.cc:5495] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ae448d0f0be4eb3ac28cf4ff915a958 (127.7.111.67). New cstate: current_term: 1 leader_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16939728527c4f3287b355862cc6a6f0" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "783323bb150f42aeb023d8577443ffc0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 39893 } health_report { overall_health: HEALTHY } } }
I20260811 12:47:15.744745 20142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:15.739477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41250 (local address 127.7.111.67:39893)
0811 12:47:15.739682 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:15.739688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:15.739801 (+ 113us) server_negotiation.cc:407] Connection header received
0811 12:47:15.739976 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:15.739981 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:15.740085 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:15.740215 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:47:15.740517 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.741239 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.742662 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.743142 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.743243 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:15.743743 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:47:15.744248 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:47:15.744361 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:15.744403 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 12:47:15.744470 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 12:47:15.753233 19774 raft_consensus.cc:1261] T e188973ab73744c782f70858223e2ee4 P 16939728527c4f3287b355862cc6a6f0 [term 1 FOLLOWER]: Refusing update from remote peer 9ae448d0f0be4eb3ac28cf4ff915a958: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:47:15.753346 19545 raft_consensus.cc:1261] T e188973ab73744c782f70858223e2ee4 P 783323bb150f42aeb023d8577443ffc0 [term 1 FOLLOWER]: Refusing update from remote peer 9ae448d0f0be4eb3ac28cf4ff915a958: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:47:15.754302 20157 consensus_queue.cc:1035] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16939728527c4f3287b355862cc6a6f0" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:47:15.754755 20155 consensus_queue.cc:1035] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958 [LEADER]: Connected to new peer: Peer: permanent_uuid: "783323bb150f42aeb023d8577443ffc0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:47:15.768231 20166 mvcc.cc:205] Tried to move back new op lower bound from 7317309176834572288 to 7317309176717348864. Current Snapshot: MvccSnapshot[applied={T|T < 7317309176834572288}]
I20260811 12:47:15.773114 20169 mvcc.cc:205] Tried to move back new op lower bound from 7317309176834572288 to 7317309176717348864. Current Snapshot: MvccSnapshot[applied={T|T < 7317309176834572288}]
I20260811 12:47:15.796125 18089 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:47:15.796475 18089 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:47:15.803790 18089 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8acd3cadf1424c2792ed3fc7e4c5f506: Sending DeleteTablet for 3 replicas of tablet e188973ab73744c782f70858223e2ee4
I20260811 12:47:15.805002 19740 tablet_service.cc:1514] Processing DeleteTablet for tablet e188973ab73744c782f70858223e2ee4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:47:15 UTC) from {username='oryx', principal='oryx/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:33820
I20260811 12:47:15.805275 19517 tablet_service.cc:1514] Processing DeleteTablet for tablet e188973ab73744c782f70858223e2ee4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:47:15 UTC) from {username='oryx', principal='oryx/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:45614
I20260811 12:47:15.805804 19985 tablet_service.cc:1514] Processing DeleteTablet for tablet e188973ab73744c782f70858223e2ee4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:47:15 UTC) from {username='oryx', principal='oryx/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:41238
I20260811 12:47:15.814764 20180 tablet_replica.cc:330] T e188973ab73744c782f70858223e2ee4 P 783323bb150f42aeb023d8577443ffc0: stopping tablet replica
I20260811 12:47:15.815451 20180 raft_consensus.cc:2229] T e188973ab73744c782f70858223e2ee4 P 783323bb150f42aeb023d8577443ffc0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:47:15.816126 20180 raft_consensus.cc:2258] T e188973ab73744c782f70858223e2ee4 P 783323bb150f42aeb023d8577443ffc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:47:15.818480 20180 ts_tablet_manager.cc:1905] T e188973ab73744c782f70858223e2ee4 P 783323bb150f42aeb023d8577443ffc0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:47:15.819418 20183 tablet_replica.cc:330] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958: stopping tablet replica
I20260811 12:47:15.819886 20183 raft_consensus.cc:2229] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:47:15.820719 20183 raft_consensus.cc:2258] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:47:15.821347 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 18041
I20260811 12:47:15.822456 18137 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:15.823176 20183 ts_tablet_manager.cc:1905] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:47:15.824245 20178 tablet_replica.cc:330] T e188973ab73744c782f70858223e2ee4 P 16939728527c4f3287b355862cc6a6f0: stopping tablet replica
I20260811 12:47:15.824739 20178 raft_consensus.cc:2229] T e188973ab73744c782f70858223e2ee4 P 16939728527c4f3287b355862cc6a6f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:47:15.825311 20178 raft_consensus.cc:2258] T e188973ab73744c782f70858223e2ee4 P 16939728527c4f3287b355862cc6a6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:47:15.828051 20180 ts_tablet_manager.cc:1918] T e188973ab73744c782f70858223e2ee4 P 783323bb150f42aeb023d8577443ffc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:47:15.828295 20180 log.cc:1196] T e188973ab73744c782f70858223e2ee4 P 783323bb150f42aeb023d8577443ffc0: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/e188973ab73744c782f70858223e2ee4
I20260811 12:47:15.828946 20180 ts_tablet_manager.cc:1939] T e188973ab73744c782f70858223e2ee4 P 783323bb150f42aeb023d8577443ffc0: Deleting consensus metadata
I20260811 12:47:15.831023 20183 ts_tablet_manager.cc:1918] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:47:15.831215 20183 log.cc:1196] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/e188973ab73744c782f70858223e2ee4
I20260811 12:47:15.831885 20183 ts_tablet_manager.cc:1939] T e188973ab73744c782f70858223e2ee4 P 9ae448d0f0be4eb3ac28cf4ff915a958: Deleting consensus metadata
I20260811 12:47:15.833618 20178 ts_tablet_manager.cc:1905] T e188973ab73744c782f70858223e2ee4 P 16939728527c4f3287b355862cc6a6f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:47:15.841060 20178 ts_tablet_manager.cc:1918] T e188973ab73744c782f70858223e2ee4 P 16939728527c4f3287b355862cc6a6f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:47:15.841244 20178 log.cc:1196] T e188973ab73744c782f70858223e2ee4 P 16939728527c4f3287b355862cc6a6f0: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/e188973ab73744c782f70858223e2ee4
I20260811 12:47:15.842098 20178 ts_tablet_manager.cc:1939] T e188973ab73744c782f70858223e2ee4 P 16939728527c4f3287b355862cc6a6f0: Deleting consensus metadata
I20260811 12:47:15.850272 20114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:15.808142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35588 (local address 127.7.111.125:38817)
0811 12:47:15.808327 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:15.808333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:15.819657 (+ 11324us) server_negotiation.cc:407] Connection header received
0811 12:47:15.831625 (+ 11968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:15.831632 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:15.831751 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:15.831852 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:15.834687 (+ 2835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.835863 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.838019 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.838649 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.838785 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:15.842706 (+ 3921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:15.842712 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:15.842718 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:15.842764 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:15.846714 (+ 3950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:15.847681 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:15.847687 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:15.847691 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:15.847856 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:15.848339 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:15.848344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:15.848347 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:15.848797 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:15.849044 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:15.849683 (+ 639us) server_negotiation.cc:299] Negotiation successful
0811 12:47:15.849923 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:47:15.869596 20116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:15.820234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40354 (local address 127.7.111.124:44773)
0811 12:47:15.820407 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:15.820412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:15.851714 (+ 31302us) server_negotiation.cc:407] Connection header received
0811 12:47:15.851990 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:15.851996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:15.852100 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:15.852176 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:15.854390 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.855715 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.857851 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:15.858494 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:15.858602 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:15.862657 (+ 4055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:15.862665 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:15.862670 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:15.862714 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:15.866418 (+ 3704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:15.867267 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:15.867273 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:15.867282 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:15.867444 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:15.867973 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:15.867979 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:15.867982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:15.868356 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:15.868619 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:15.868975 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 12:47:15.869219 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 12:47:15.969779 18062 catalog_manager.cc:4841] TS 16939728527c4f3287b355862cc6a6f0 (127.7.111.66:46081): tablet e188973ab73744c782f70858223e2ee4 (table test-table [id=434d576bdee3498f9804c9a8460ae783]) successfully deleted
I20260811 12:47:15.971259 18063 catalog_manager.cc:4841] TS 783323bb150f42aeb023d8577443ffc0 (127.7.111.65:33335): tablet e188973ab73744c782f70858223e2ee4 (table test-table [id=434d576bdee3498f9804c9a8460ae783]) successfully deleted
I20260811 12:47:15.971273 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 18041
W20260811 12:47:15.999377 19903 connection.cc:535] server connection from 127.0.0.1:41238 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 12:47:16.034248 20114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:16.010065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35594 (local address 127.7.111.125:38817)
0811 12:47:16.010291 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:16.010299 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:16.017481 (+ 7182us) server_negotiation.cc:407] Connection header received
0811 12:47:16.017806 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:16.017812 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:16.017901 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:16.017978 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:16.020366 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:16.021533 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:16.023620 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:16.024307 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:16.024410 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:16.028582 (+ 4172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:16.028587 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:16.028590 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:16.028627 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:16.031393 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:16.032022 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:16.032028 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:16.032031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:16.032183 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:16.032629 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:16.032634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:16.032637 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:16.033003 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:16.033264 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:16.033665 (+ 401us) server_negotiation.cc:299] Negotiation successful
0811 12:47:16.033914 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:47:16.036054 20116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:16.010121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40366 (local address 127.7.111.124:44773)
0811 12:47:16.010300 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:16.010306 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:16.017479 (+ 7173us) server_negotiation.cc:407] Connection header received
0811 12:47:16.017847 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:16.017852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:16.017929 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:16.017991 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:16.020365 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:16.021760 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:16.023657 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:16.024316 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:16.024411 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:16.028558 (+ 4147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:16.028565 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:16.028571 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:16.028620 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:16.031989 (+ 3369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:16.032685 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:16.032690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:16.032694 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:16.032839 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:16.033264 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:16.033269 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:16.033272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:16.033808 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:16.034032 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:16.035599 (+ 1567us) server_negotiation.cc:299] Negotiation successful
0811 12:47:16.035804 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:47:16.780633 20197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:16.779989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.66:53527)
0811 12:47:16.780383 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:16.780489 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":193,"threads_started":1}
W20260811 12:47:16.781098 19852 heartbeater.cc:644] Failed to heartbeat to 127.7.111.126:34613 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
I20260811 12:47:16.789324 20199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:16.788227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.67:48837)
0811 12:47:16.788874 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:16.789088 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":547,"threads_started":1}
I20260811 12:47:16.789624 20198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:16.787606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.65:34557)
0811 12:47:16.789262 (+ 1656us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:16.789430 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1525,"thread_start_us":1760,"threads_started":1}
W20260811 12:47:16.789710 20095 heartbeater.cc:644] Failed to heartbeat to 127.7.111.126:34613 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
W20260811 12:47:16.790050 19619 heartbeater.cc:644] Failed to heartbeat to 127.7.111.126:34613 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
I20260811 12:47:17.304219 20201 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8acd3cadf1424c2792ed3fc7e4c5f506)
I20260811 12:47:17.304195 20200 raft_consensus.cc:482] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8acd3cadf1424c2792ed3fc7e4c5f506)
I20260811 12:47:17.304491 20201 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: false } }
I20260811 12:47:17.304498 20200 raft_consensus.cc:504] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: false } }
I20260811 12:47:17.305961 20200 leader_election.cc:288] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8acd3cadf1424c2792ed3fc7e4c5f506 (127.7.111.126:34613), 88bd75212b2e4308a018df055d5b6ccd (127.7.111.124:44773)
I20260811 12:47:17.305999 20201 leader_election.cc:288] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8acd3cadf1424c2792ed3fc7e4c5f506 (127.7.111.126:34613), 148ff30033374e829df5a9d57fb0ddba (127.7.111.125:38817)
I20260811 12:47:17.315304 20204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:17.306136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.0.0.1:57134)
0811 12:47:17.314800 (+ 8664us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:17.314980 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7017,"spinlock_wait_cycles":196480,"thread_start_us":7086,"threads_started":1}
W20260811 12:47:17.315692 19307 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
I20260811 12:47:17.321568 20206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:17.320160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.0.0.1:57158)
0811 12:47:17.321145 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:17.321321 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":859}
I20260811 12:47:17.321964 20202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:17.306091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.0.0.1:57136)
0811 12:47:17.321482 (+ 15391us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:17.321664 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15253,"threads_started":1}
W20260811 12:47:17.321976 19307 leader_election.cc:334] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8acd3cadf1424c2792ed3fc7e4c5f506 (127.7.111.126:34613): Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
W20260811 12:47:17.322700 18998 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
I20260811 12:47:17.325675 20208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:17.325159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.0.0.1:57172)
0811 12:47:17.325351 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:17.325526 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 12:47:17.325834 18998 leader_election.cc:334] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8acd3cadf1424c2792ed3fc7e4c5f506 (127.7.111.126:34613): Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
I20260811 12:47:17.328823 20212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:17.322111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35598 (local address 127.7.111.125:38817)
0811 12:47:17.322787 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:17.322793 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:17.322814 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:47:17.322898 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:17.322903 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:17.322976 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:17.323077 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:47:17.323590 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:17.324875 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:17.326807 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:17.327960 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:17.328053 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:17.328124 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:17.328492 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 12:47:17.328579 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":531,"thread_start_us":338,"threads_started":1}
I20260811 12:47:17.328891 20204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:17.306474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:38817 (local address 127.0.0.1:35598)
0811 12:47:17.316315 (+ 9841us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:17.316341 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:17.316491 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:17.323133 (+ 6642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:17.323140 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:17.323186 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:47:17.323444 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:17.323455 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:17.325015 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:17.325020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:17.326652 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:17.326665 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:17.328119 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:17.328125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:17.328341 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:17.328378 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 12:47:17.328483 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 12:47:17.328625 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9745,"thread_start_us":15225,"threads_started":1}
I20260811 12:47:17.329643 19043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "88bd75212b2e4308a018df055d5b6ccd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "148ff30033374e829df5a9d57fb0ddba" is_pre_election: true
I20260811 12:47:17.329959 20210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:17.306952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40372 (local address 127.7.111.124:44773)
0811 12:47:17.323871 (+ 16919us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:17.323875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:17.323889 (+ 14us) server_negotiation.cc:407] Connection header received
0811 12:47:17.323930 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:17.323933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:17.323980 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:17.324066 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:47:17.324689 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:17.325806 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:17.327790 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:17.328885 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:17.329005 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:17.329113 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:17.329582 (+ 469us) server_negotiation.cc:299] Negotiation successful
0811 12:47:17.329694 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":16826,"thread_start_us":16708,"threads_started":1}
I20260811 12:47:17.329959 20202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:17.306480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:44773 (local address 127.0.0.1:40372)
0811 12:47:17.322159 (+ 15679us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:17.322189 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:17.322324 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:17.324114 (+ 1790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:17.324119 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:17.324145 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:47:17.324496 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:17.324511 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:17.325981 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:17.325986 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:17.327631 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:17.327646 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:17.329235 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:17.329240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:17.329423 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:17.329455 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:47:17.329551 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 12:47:17.329703 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15594,"thread_start_us":17934,"threads_started":1}
I20260811 12:47:17.330029 19043 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88bd75212b2e4308a018df055d5b6ccd in term 1.
I20260811 12:47:17.330562 19311 leader_election.cc:302] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 148ff30033374e829df5a9d57fb0ddba, 88bd75212b2e4308a018df055d5b6ccd; no voters: 8acd3cadf1424c2792ed3fc7e4c5f506
I20260811 12:47:17.330613 19345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "148ff30033374e829df5a9d57fb0ddba" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "88bd75212b2e4308a018df055d5b6ccd" is_pre_election: true
I20260811 12:47:17.330924 19345 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 148ff30033374e829df5a9d57fb0ddba in term 1.
I20260811 12:47:17.331094 20201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 12:47:17.331234 20201 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8acd3cadf1424c2792ed3fc7e4c5f506)
I20260811 12:47:17.331346 20201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 1 FOLLOWER]: Advancing to term 2
I20260811 12:47:17.331411 18995 leader_election.cc:302] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 148ff30033374e829df5a9d57fb0ddba, 88bd75212b2e4308a018df055d5b6ccd; no voters: 8acd3cadf1424c2792ed3fc7e4c5f506
I20260811 12:47:17.331789 20200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 12:47:17.331934 20200 raft_consensus.cc:482] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8acd3cadf1424c2792ed3fc7e4c5f506)
I20260811 12:47:17.332031 20200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 1 FOLLOWER]: Advancing to term 2
I20260811 12:47:17.333674 20201 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: false } }
I20260811 12:47:17.334172 20200 raft_consensus.cc:504] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: false } }
I20260811 12:47:17.334323 20201 leader_election.cc:288] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [CANDIDATE]: Term 2 election: Requested vote from peers 8acd3cadf1424c2792ed3fc7e4c5f506 (127.7.111.126:34613), 148ff30033374e829df5a9d57fb0ddba (127.7.111.125:38817)
I20260811 12:47:17.334874 20200 leader_election.cc:288] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [CANDIDATE]: Term 2 election: Requested vote from peers 8acd3cadf1424c2792ed3fc7e4c5f506 (127.7.111.126:34613), 88bd75212b2e4308a018df055d5b6ccd (127.7.111.124:44773)
I20260811 12:47:17.334856 19043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "88bd75212b2e4308a018df055d5b6ccd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "148ff30033374e829df5a9d57fb0ddba"
I20260811 12:47:17.335101 20204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:17.334416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.0.0.1:57188)
0811 12:47:17.334851 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:17.334939 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324}
I20260811 12:47:17.335248 19043 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 88bd75212b2e4308a018df055d5b6ccd in current term 2: Already voted for candidate 148ff30033374e829df5a9d57fb0ddba in this term.
I20260811 12:47:17.335343 20202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:17.334973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.0.0.1:57192)
0811 12:47:17.335108 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:17.335193 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 12:47:17.335637 19345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "148ff30033374e829df5a9d57fb0ddba" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "88bd75212b2e4308a018df055d5b6ccd"
I20260811 12:47:17.335927 19345 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 148ff30033374e829df5a9d57fb0ddba in current term 2: Already voted for candidate 88bd75212b2e4308a018df055d5b6ccd in this term.
I20260811 12:47:17.336454 20204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:17.336058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.0.0.1:57198)
0811 12:47:17.336243 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:17.336330 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 12:47:17.336514 20202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:17.336119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.0.0.1:57212)
0811 12:47:17.336305 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:17.336396 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260811 12:47:17.336642 19307 leader_election.cc:334] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8acd3cadf1424c2792ed3fc7e4c5f506 (127.7.111.126:34613): Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
W20260811 12:47:17.336647 18998 leader_election.cc:334] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8acd3cadf1424c2792ed3fc7e4c5f506 (127.7.111.126:34613): Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
I20260811 12:47:17.336762 18998 leader_election.cc:302] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 148ff30033374e829df5a9d57fb0ddba; no voters: 88bd75212b2e4308a018df055d5b6ccd, 8acd3cadf1424c2792ed3fc7e4c5f506
I20260811 12:47:17.336766 19307 leader_election.cc:302] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88bd75212b2e4308a018df055d5b6ccd; no voters: 148ff30033374e829df5a9d57fb0ddba, 8acd3cadf1424c2792ed3fc7e4c5f506
I20260811 12:47:17.337029 20200 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 12:47:17.337282 20201 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 12:47:17.782470 20215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:17.781761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.66:34369)
0811 12:47:17.782202 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:17.782288 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":225,"threads_started":1}
I20260811 12:47:17.791815 20217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:17.790703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.65:55421)
0811 12:47:17.791229 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:17.791355 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":298,"threads_started":1}
I20260811 12:47:17.793980 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:17.790419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.67:51045)
0811 12:47:17.793736 (+ 3317us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:17.793822 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3220,"thread_start_us":3116,"threads_started":1}
I20260811 12:47:18.783923 20221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:18.783214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.66:58781)
0811 12:47:18.783643 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:18.783748 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":218,"threads_started":1}
I20260811 12:47:18.793617 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:18.792746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.65:48443)
0811 12:47:18.793225 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:18.793364 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":241,"threads_started":1}
I20260811 12:47:18.795396 20226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:18.794666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.67:60755)
0811 12:47:18.795130 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:18.795222 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":256,"threads_started":1}
I20260811 12:47:19.048033 20227 raft_consensus.cc:482] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:47:19.048302 20227 raft_consensus.cc:504] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: false } }
I20260811 12:47:19.049017 20227 leader_election.cc:288] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8acd3cadf1424c2792ed3fc7e4c5f506 (127.7.111.126:34613), 88bd75212b2e4308a018df055d5b6ccd (127.7.111.124:44773)
I20260811 12:47:19.049772 19345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "148ff30033374e829df5a9d57fb0ddba" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "88bd75212b2e4308a018df055d5b6ccd" is_pre_election: true
I20260811 12:47:19.050071 19345 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 148ff30033374e829df5a9d57fb0ddba in term 2.
I20260811 12:47:19.050179 20230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.049235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.0.0.1:57440)
0811 12:47:19.049797 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.049942 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":263,"threads_started":1}
I20260811 12:47:19.050595 18995 leader_election.cc:302] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148ff30033374e829df5a9d57fb0ddba, 88bd75212b2e4308a018df055d5b6ccd; no voters:
I20260811 12:47:19.050880 20227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 12:47:19.050999 20227 raft_consensus.cc:482] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:47:19.051097 20227 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 2 FOLLOWER]: Advancing to term 3
I20260811 12:47:19.051684 20230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.051352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.0.0.1:57446)
0811 12:47:19.051472 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.051565 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 12:47:19.051882 18998 leader_election.cc:334] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8acd3cadf1424c2792ed3fc7e4c5f506 (127.7.111.126:34613): Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
I20260811 12:47:19.053931 20227 raft_consensus.cc:504] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: false } }
I20260811 12:47:19.054627 20227 leader_election.cc:288] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [CANDIDATE]: Term 3 election: Requested vote from peers 8acd3cadf1424c2792ed3fc7e4c5f506 (127.7.111.126:34613), 88bd75212b2e4308a018df055d5b6ccd (127.7.111.124:44773)
I20260811 12:47:19.054910 19345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "148ff30033374e829df5a9d57fb0ddba" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "88bd75212b2e4308a018df055d5b6ccd"
I20260811 12:47:19.055015 20230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.054567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.0.0.1:57454)
0811 12:47:19.054802 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.054881 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260811 12:47:19.055119 19345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 2 FOLLOWER]: Advancing to term 3
I20260811 12:47:19.055835 20230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.055525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.0.0.1:57458)
0811 12:47:19.055661 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.055727 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 12:47:19.055972 18998 leader_election.cc:334] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 8acd3cadf1424c2792ed3fc7e4c5f506 (127.7.111.126:34613): Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
I20260811 12:47:19.057421 19345 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 148ff30033374e829df5a9d57fb0ddba in term 3.
I20260811 12:47:19.057783 18995 leader_election.cc:302] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 148ff30033374e829df5a9d57fb0ddba, 88bd75212b2e4308a018df055d5b6ccd; no voters: 8acd3cadf1424c2792ed3fc7e4c5f506
I20260811 12:47:19.058053 20227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 3 FOLLOWER]: Leader election won for term 3
I20260811 12:47:19.058432 20227 raft_consensus.cc:686] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [term 3 LEADER]: Becoming Leader. State: Replica: 148ff30033374e829df5a9d57fb0ddba, State: Running, Role: LEADER
I20260811 12:47:19.058782 20227 consensus_queue.cc:237] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: false } }
I20260811 12:47:19.060375 20232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 148ff30033374e829df5a9d57fb0ddba. Latest consensus state: current_term: 3 leader_uuid: "148ff30033374e829df5a9d57fb0ddba" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: false } } }
I20260811 12:47:19.060638 20232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:19.061209 20236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:47:19.065105 20236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:47:19.066049 20236 catalog_manager.cc:1259] Loaded cluster ID: 53e0569ae84d4f13b3a2305a0117f25e
I20260811 12:47:19.066123 20236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:47:19.067256 20236 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:47:19.068064 20236 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba: Loaded TSK: 0
I20260811 12:47:19.068876 20236 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:47:19.141126 19028 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35594:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:47:19.148545 19028 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:47:19.152817 20230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.152415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.0.0.1:57466)
0811 12:47:19.152584 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.152680 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 12:47:19.152925 19345 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [term 3 FOLLOWER]: Refusing update from remote peer 148ff30033374e829df5a9d57fb0ddba: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 12:47:19.153734 20232 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [LEADER]: Connected to new peer: Peer: permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 12:47:19.153965 20230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.153626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.0.0.1:57480)
0811 12:47:19.153779 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.153851 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260811 12:47:19.154163 18998 consensus_peers.cc:487] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba -> Peer 8acd3cadf1424c2792ed3fc7e4c5f506 (127.7.111.126:34613): Couldn't send request to peer 8acd3cadf1424c2792ed3fc7e4c5f506. Status: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 12:47:19.156711 20239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 148ff30033374e829df5a9d57fb0ddba. Latest consensus state: current_term: 3 leader_uuid: "148ff30033374e829df5a9d57fb0ddba" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: false } } }
I20260811 12:47:19.156960 20239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:47:19.158154 20232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "148ff30033374e829df5a9d57fb0ddba" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: false } } }
I20260811 12:47:19.158438 20232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:19.158818 20239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "148ff30033374e829df5a9d57fb0ddba" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8acd3cadf1424c2792ed3fc7e4c5f506" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34613 } } peers { permanent_uuid: "148ff30033374e829df5a9d57fb0ddba" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 38817 } attrs { promote: false } } peers { permanent_uuid: "88bd75212b2e4308a018df055d5b6ccd" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 44773 } attrs { promote: false } } }
I20260811 12:47:19.159070 20239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88bd75212b2e4308a018df055d5b6ccd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:47:19.160146 20240 mvcc.cc:205] Tried to move back new op lower bound from 7317309190763040768 to 7317309190387961856. Current Snapshot: MvccSnapshot[applied={T|T < 7317309190763040768 or (T in {7317309190763040768})}]
I20260811 12:47:19.173202 20254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.167157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45620 (local address 127.7.111.65:33335)
0811 12:47:19.167629 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:19.167633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:19.167649 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:47:19.167723 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:19.167728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:19.167799 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:19.167897 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:47:19.168636 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:19.169340 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:19.171347 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:19.172229 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:19.172347 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:19.172450 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:19.172751 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 12:47:19.172889 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":850,"threads_started":1}
I20260811 12:47:19.173236 20250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.166730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:33335 (local address 127.0.0.1:45620)
0811 12:47:19.167293 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.167312 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:19.167420 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:19.168112 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:19.168117 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:19.168147 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:47:19.168473 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:19.168480 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:19.169529 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:19.169533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:19.171155 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:19.171169 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:19.172386 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:19.172391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:19.172602 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:19.172628 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 12:47:19.172708 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 12:47:19.172830 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":1,"thread_start_us":1381,"threads_started":1}
I20260811 12:47:19.174482 20230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.166375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:39893 (local address 127.0.0.1:41260)
0811 12:47:19.166529 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.166548 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:19.166683 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:19.167792 (+ 1109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:19.167797 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:19.167829 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:47:19.168193 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:19.168211 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:19.169874 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:19.169883 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:19.172023 (+ 2140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:19.172039 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:19.173841 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:19.173847 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:19.173975 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:19.173999 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 12:47:19.174078 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 12:47:19.174197 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 12:47:19.174606 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.166486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41260 (local address 127.7.111.67:39893)
0811 12:47:19.167432 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:19.167438 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:19.167457 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:47:19.167528 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:19.167533 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:19.167612 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:19.168352 (+ 740us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:47:19.168591 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:19.169689 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:19.172265 (+ 2576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:19.173601 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:19.173778 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:19.173890 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:19.174113 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 12:47:19.174253 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":815,"thread_start_us":694,"threads_started":1}
I20260811 12:47:19.174980 19517 tablet_service.cc:1467] Processing CreateTablet for tablet a144caea5d504a858e70809f04daddbb (DEFAULT_TABLE table=test-table [id=0adfcbdfa71f47c2aa1e9ff32975ad4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:47:19.175397 19517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a144caea5d504a858e70809f04daddbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:19.175302 19985 tablet_service.cc:1467] Processing CreateTablet for tablet a144caea5d504a858e70809f04daddbb (DEFAULT_TABLE table=test-table [id=0adfcbdfa71f47c2aa1e9ff32975ad4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:47:19.175752 19985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a144caea5d504a858e70809f04daddbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:19.181094 20261 tablet_bootstrap.cc:492] T a144caea5d504a858e70809f04daddbb P 783323bb150f42aeb023d8577443ffc0: Bootstrap starting.
I20260811 12:47:19.182585 20261 tablet_bootstrap.cc:654] T a144caea5d504a858e70809f04daddbb P 783323bb150f42aeb023d8577443ffc0: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:19.185256 20259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.167516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:46081 (local address 127.0.0.1:33828)
0811 12:47:19.168080 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.168095 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:19.168210 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:19.178645 (+ 10435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:19.178650 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:19.178675 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:47:19.179040 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:19.179051 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:19.180476 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:19.180481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:19.182651 (+ 2170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:19.182668 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:19.184406 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:19.184413 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:19.184600 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:19.184636 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 12:47:19.184747 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 12:47:19.184887 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":720,"threads_started":1}
I20260811 12:47:19.185353 20260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.167894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33828 (local address 127.7.111.66:46081)
0811 12:47:19.178115 (+ 10221us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:19.178121 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:19.178140 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:47:19.178218 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:19.178245 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:19.178319 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:19.178454 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:47:19.179260 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:19.180319 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:19.182810 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:19.184001 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:19.184153 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:19.184264 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:19.184807 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 12:47:19.184973 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10078,"thread_start_us":9951,"threads_started":1}
I20260811 12:47:19.186424 19740 tablet_service.cc:1467] Processing CreateTablet for tablet a144caea5d504a858e70809f04daddbb (DEFAULT_TABLE table=test-table [id=0adfcbdfa71f47c2aa1e9ff32975ad4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:47:19.186918 19740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a144caea5d504a858e70809f04daddbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:19.190388 20263 tablet_bootstrap.cc:492] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958: Bootstrap starting.
I20260811 12:47:19.191481 20261 tablet_bootstrap.cc:492] T a144caea5d504a858e70809f04daddbb P 783323bb150f42aeb023d8577443ffc0: No bootstrap required, opened a new log
I20260811 12:47:19.191628 20261 ts_tablet_manager.cc:1397] T a144caea5d504a858e70809f04daddbb P 783323bb150f42aeb023d8577443ffc0: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260811 12:47:19.192267 20263 tablet_bootstrap.cc:654] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:19.192440 20261 raft_consensus.cc:348] T a144caea5d504a858e70809f04daddbb P 783323bb150f42aeb023d8577443ffc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 39893 } } peers { permanent_uuid: "783323bb150f42aeb023d8577443ffc0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33335 } } peers { permanent_uuid: "16939728527c4f3287b355862cc6a6f0" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46081 } }
I20260811 12:47:19.192687 20261 raft_consensus.cc:374] T a144caea5d504a858e70809f04daddbb P 783323bb150f42aeb023d8577443ffc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:19.192827 20261 raft_consensus.cc:729] T a144caea5d504a858e70809f04daddbb P 783323bb150f42aeb023d8577443ffc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 783323bb150f42aeb023d8577443ffc0, State: Initialized, Role: FOLLOWER
I20260811 12:47:19.193087 20261 consensus_queue.cc:260] T a144caea5d504a858e70809f04daddbb P 783323bb150f42aeb023d8577443ffc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 39893 } } peers { permanent_uuid: "783323bb150f42aeb023d8577443ffc0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33335 } } peers { permanent_uuid: "16939728527c4f3287b355862cc6a6f0" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46081 } }
I20260811 12:47:19.194284 20263 tablet_bootstrap.cc:492] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958: No bootstrap required, opened a new log
I20260811 12:47:19.194422 20263 ts_tablet_manager.cc:1397] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 12:47:19.195008 20263 raft_consensus.cc:348] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 39893 } } peers { permanent_uuid: "783323bb150f42aeb023d8577443ffc0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33335 } } peers { permanent_uuid: "16939728527c4f3287b355862cc6a6f0" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46081 } }
I20260811 12:47:19.195209 20263 raft_consensus.cc:374] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:19.195338 20263 raft_consensus.cc:729] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ae448d0f0be4eb3ac28cf4ff915a958, State: Initialized, Role: FOLLOWER
I20260811 12:47:19.195546 20263 consensus_queue.cc:260] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 39893 } } peers { permanent_uuid: "783323bb150f42aeb023d8577443ffc0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33335 } } peers { permanent_uuid: "16939728527c4f3287b355862cc6a6f0" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46081 } }
I20260811 12:47:19.196291 20263 ts_tablet_manager.cc:1428] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 12:47:19.196388 20269 tablet_bootstrap.cc:492] T a144caea5d504a858e70809f04daddbb P 16939728527c4f3287b355862cc6a6f0: Bootstrap starting.
I20260811 12:47:19.197626 20226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.197093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.67:38407)
0811 12:47:19.197351 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.197480 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260811 12:47:19.199064 20226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.198626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.67:47473)
0811 12:47:19.198832 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.198909 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260811 12:47:19.199937 20269 tablet_bootstrap.cc:654] T a144caea5d504a858e70809f04daddbb P 16939728527c4f3287b355862cc6a6f0: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:19.199934 20097 heartbeater.cc:503] Master 127.7.111.125:38817 was elected leader, sending a full tablet report...
I20260811 12:47:19.202586 20261 ts_tablet_manager.cc:1428] T a144caea5d504a858e70809f04daddbb P 783323bb150f42aeb023d8577443ffc0: Time spent starting tablet: real 0.011s user 0.003s sys 0.007s
I20260811 12:47:19.203442 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.202961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.65:55745)
0811 12:47:19.203170 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.203289 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260811 12:47:19.203558 20269 tablet_bootstrap.cc:492] T a144caea5d504a858e70809f04daddbb P 16939728527c4f3287b355862cc6a6f0: No bootstrap required, opened a new log
I20260811 12:47:19.203701 20269 ts_tablet_manager.cc:1397] T a144caea5d504a858e70809f04daddbb P 16939728527c4f3287b355862cc6a6f0: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260811 12:47:19.204394 20269 raft_consensus.cc:348] T a144caea5d504a858e70809f04daddbb P 16939728527c4f3287b355862cc6a6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 39893 } } peers { permanent_uuid: "783323bb150f42aeb023d8577443ffc0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33335 } } peers { permanent_uuid: "16939728527c4f3287b355862cc6a6f0" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46081 } }
I20260811 12:47:19.204632 20269 raft_consensus.cc:374] T a144caea5d504a858e70809f04daddbb P 16939728527c4f3287b355862cc6a6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:19.204728 20269 raft_consensus.cc:729] T a144caea5d504a858e70809f04daddbb P 16939728527c4f3287b355862cc6a6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16939728527c4f3287b355862cc6a6f0, State: Initialized, Role: FOLLOWER
I20260811 12:47:19.204999 20269 consensus_queue.cc:260] T a144caea5d504a858e70809f04daddbb P 16939728527c4f3287b355862cc6a6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 39893 } } peers { permanent_uuid: "783323bb150f42aeb023d8577443ffc0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33335 } } peers { permanent_uuid: "16939728527c4f3287b355862cc6a6f0" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46081 } }
I20260811 12:47:19.205188 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.204788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.65:59677)
0811 12:47:19.204971 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.205046 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 12:47:19.205865 20269 ts_tablet_manager.cc:1428] T a144caea5d504a858e70809f04daddbb P 16939728527c4f3287b355862cc6a6f0: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260811 12:47:19.206626 19620 heartbeater.cc:503] Master 127.7.111.125:38817 was elected leader, sending a full tablet report...
I20260811 12:47:19.207762 20221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.207244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.66:48303)
0811 12:47:19.207520 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.207616 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180}
I20260811 12:47:19.208238 19853 heartbeater.cc:503] Master 127.7.111.125:38817 was elected leader, sending a full tablet report...
I20260811 12:47:19.208655 20221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.208308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.66:54353)
0811 12:47:19.208456 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.208528 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 12:47:19.286000 19865 tablet.cc:2369] T a144caea5d504a858e70809f04daddbb P 16939728527c4f3287b355862cc6a6f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:47:19.407581 20273 raft_consensus.cc:482] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:47:19.407821 20273 raft_consensus.cc:504] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 39893 } } peers { permanent_uuid: "783323bb150f42aeb023d8577443ffc0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33335 } } peers { permanent_uuid: "16939728527c4f3287b355862cc6a6f0" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46081 } }
I20260811 12:47:19.408532 20273 leader_election.cc:288] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 783323bb150f42aeb023d8577443ffc0 (127.7.111.65:33335), 16939728527c4f3287b355862cc6a6f0 (127.7.111.66:46081)
I20260811 12:47:19.409075 19544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a144caea5d504a858e70809f04daddbb" candidate_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "783323bb150f42aeb023d8577443ffc0" is_pre_election: true
I20260811 12:47:19.409130 19772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a144caea5d504a858e70809f04daddbb" candidate_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16939728527c4f3287b355862cc6a6f0" is_pre_election: true
I20260811 12:47:19.409370 19544 raft_consensus.cc:2454] T a144caea5d504a858e70809f04daddbb P 783323bb150f42aeb023d8577443ffc0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ae448d0f0be4eb3ac28cf4ff915a958 in term 0.
I20260811 12:47:19.409423 19772 raft_consensus.cc:2454] T a144caea5d504a858e70809f04daddbb P 16939728527c4f3287b355862cc6a6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ae448d0f0be4eb3ac28cf4ff915a958 in term 0.
I20260811 12:47:19.409861 19904 leader_election.cc:302] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 16939728527c4f3287b355862cc6a6f0, 783323bb150f42aeb023d8577443ffc0, 9ae448d0f0be4eb3ac28cf4ff915a958; no voters:
I20260811 12:47:19.410110 20273 raft_consensus.cc:2781] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:47:19.410218 20273 raft_consensus.cc:482] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:47:19.410305 20273 raft_consensus.cc:3037] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:19.412535 20273 raft_consensus.cc:504] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 39893 } } peers { permanent_uuid: "783323bb150f42aeb023d8577443ffc0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33335 } } peers { permanent_uuid: "16939728527c4f3287b355862cc6a6f0" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46081 } }
I20260811 12:47:19.413038 20273 leader_election.cc:288] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958 [CANDIDATE]: Term 1 election: Requested vote from peers 783323bb150f42aeb023d8577443ffc0 (127.7.111.65:33335), 16939728527c4f3287b355862cc6a6f0 (127.7.111.66:46081)
I20260811 12:47:19.413508 19544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a144caea5d504a858e70809f04daddbb" candidate_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "783323bb150f42aeb023d8577443ffc0"
I20260811 12:47:19.413558 19772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a144caea5d504a858e70809f04daddbb" candidate_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16939728527c4f3287b355862cc6a6f0"
I20260811 12:47:19.413687 19544 raft_consensus.cc:3037] T a144caea5d504a858e70809f04daddbb P 783323bb150f42aeb023d8577443ffc0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:19.413725 19772 raft_consensus.cc:3037] T a144caea5d504a858e70809f04daddbb P 16939728527c4f3287b355862cc6a6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:19.415704 19772 raft_consensus.cc:2454] T a144caea5d504a858e70809f04daddbb P 16939728527c4f3287b355862cc6a6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ae448d0f0be4eb3ac28cf4ff915a958 in term 1.
I20260811 12:47:19.415720 19544 raft_consensus.cc:2454] T a144caea5d504a858e70809f04daddbb P 783323bb150f42aeb023d8577443ffc0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ae448d0f0be4eb3ac28cf4ff915a958 in term 1.
I20260811 12:47:19.416106 19903 leader_election.cc:302] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 16939728527c4f3287b355862cc6a6f0, 783323bb150f42aeb023d8577443ffc0, 9ae448d0f0be4eb3ac28cf4ff915a958; no voters:
I20260811 12:47:19.416391 20273 raft_consensus.cc:2781] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:47:19.416816 20273 raft_consensus.cc:686] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 1 LEADER]: Becoming Leader. State: Replica: 9ae448d0f0be4eb3ac28cf4ff915a958, State: Running, Role: LEADER
I20260811 12:47:19.417096 20273 consensus_queue.cc:237] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 39893 } } peers { permanent_uuid: "783323bb150f42aeb023d8577443ffc0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33335 } } peers { permanent_uuid: "16939728527c4f3287b355862cc6a6f0" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46081 } }
I20260811 12:47:19.417824 20226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.417437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.67:35241)
0811 12:47:19.417591 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.417671 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 12:47:19.423385 19030 catalog_manager.cc:5495] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ae448d0f0be4eb3ac28cf4ff915a958 (127.7.111.67). New cstate: current_term: 1 leader_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae448d0f0be4eb3ac28cf4ff915a958" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 39893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "783323bb150f42aeb023d8577443ffc0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16939728527c4f3287b355862cc6a6f0" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46081 } health_report { overall_health: UNKNOWN } } }
I20260811 12:47:19.469792 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.465628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41270 (local address 127.7.111.67:39893)
0811 12:47:19.465817 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:19.465821 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:19.465839 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:47:19.465920 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:19.465923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:19.465988 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:19.466054 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:47:19.466351 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:19.467075 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:19.468292 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:19.468690 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:19.468788 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:19.469151 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:47:19.469264 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:47:19.469339 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:19.469452 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 12:47:19.469516 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 12:47:19.472857 19772 raft_consensus.cc:1261] T a144caea5d504a858e70809f04daddbb P 16939728527c4f3287b355862cc6a6f0 [term 1 FOLLOWER]: Refusing update from remote peer 9ae448d0f0be4eb3ac28cf4ff915a958: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:47:19.472883 19544 raft_consensus.cc:1261] T a144caea5d504a858e70809f04daddbb P 783323bb150f42aeb023d8577443ffc0 [term 1 FOLLOWER]: Refusing update from remote peer 9ae448d0f0be4eb3ac28cf4ff915a958: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:47:19.473501 20278 consensus_queue.cc:1035] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958 [LEADER]: Connected to new peer: Peer: permanent_uuid: "783323bb150f42aeb023d8577443ffc0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:47:19.473834 20273 consensus_queue.cc:1035] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16939728527c4f3287b355862cc6a6f0" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:47:19.476308 20221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.475832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.66:60521)
0811 12:47:19.476064 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.476166 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260811 12:47:19.476339 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.475828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.65:37473)
0811 12:47:19.476066 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.476164 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260811 12:47:19.481653 20221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.480981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.66:50049)
0811 12:47:19.481332 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.481444 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222}
I20260811 12:47:19.486352 20226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.485429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.67:36205)
0811 12:47:19.486031 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.486133 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483}
I20260811 12:47:19.495469 20226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.495073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.67:34111)
0811 12:47:19.495237 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.495355 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 12:47:19.495997 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.493915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.7.111.65:39915)
0811 12:47:19.494355 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.494428 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349}
I20260811 12:47:19.496438 19028 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:47:19.496742 19028 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:47:19.501935 19028 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 148ff30033374e829df5a9d57fb0ddba: Sending DeleteTablet for 3 replicas of tablet a144caea5d504a858e70809f04daddbb
I20260811 12:47:19.503160 19985 tablet_service.cc:1514] Processing DeleteTablet for tablet a144caea5d504a858e70809f04daddbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:47:19 UTC) from {username='oryx', principal='oryx/127.7.111.125@KRBTEST.COM'} at 127.0.0.1:41260
I20260811 12:47:19.503703 19517 tablet_service.cc:1514] Processing DeleteTablet for tablet a144caea5d504a858e70809f04daddbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:47:19 UTC) from {username='oryx', principal='oryx/127.7.111.125@KRBTEST.COM'} at 127.0.0.1:45620
I20260811 12:47:19.503868 20295 tablet_replica.cc:330] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958: stopping tablet replica
I20260811 12:47:19.504089 20295 raft_consensus.cc:2229] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:47:19.504628 20295 raft_consensus.cc:2258] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:47:19.505800 20295 ts_tablet_manager.cc:1905] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:47:19.506791 19740 tablet_service.cc:1514] Processing DeleteTablet for tablet a144caea5d504a858e70809f04daddbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:47:19 UTC) from {username='oryx', principal='oryx/127.7.111.125@KRBTEST.COM'} at 127.0.0.1:33828
I20260811 12:47:19.511070 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 19295
I20260811 12:47:19.513300 20295 ts_tablet_manager.cc:1918] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:47:19.513463 20295 log.cc:1196] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/a144caea5d504a858e70809f04daddbb
I20260811 12:47:19.513989 20295 ts_tablet_manager.cc:1939] T a144caea5d504a858e70809f04daddbb P 9ae448d0f0be4eb3ac28cf4ff915a958: Deleting consensus metadata
I20260811 12:47:19.515164 18997 catalog_manager.cc:4841] TS 9ae448d0f0be4eb3ac28cf4ff915a958 (127.7.111.67:39893): tablet a144caea5d504a858e70809f04daddbb (table test-table [id=0adfcbdfa71f47c2aa1e9ff32975ad4e]) successfully deleted
I20260811 12:47:19.516865 20296 tablet_replica.cc:330] T a144caea5d504a858e70809f04daddbb P 783323bb150f42aeb023d8577443ffc0: stopping tablet replica
I20260811 12:47:19.517051 20296 raft_consensus.cc:2229] T a144caea5d504a858e70809f04daddbb P 783323bb150f42aeb023d8577443ffc0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:47:19.517341 20296 raft_consensus.cc:2258] T a144caea5d504a858e70809f04daddbb P 783323bb150f42aeb023d8577443ffc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:47:19.518347 20296 ts_tablet_manager.cc:1905] T a144caea5d504a858e70809f04daddbb P 783323bb150f42aeb023d8577443ffc0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:47:19.521822 20298 tablet_replica.cc:330] T a144caea5d504a858e70809f04daddbb P 16939728527c4f3287b355862cc6a6f0: stopping tablet replica
I20260811 12:47:19.522017 20298 raft_consensus.cc:2229] T a144caea5d504a858e70809f04daddbb P 16939728527c4f3287b355862cc6a6f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:47:19.522306 20298 raft_consensus.cc:2258] T a144caea5d504a858e70809f04daddbb P 16939728527c4f3287b355862cc6a6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:47:19.523232 20298 ts_tablet_manager.cc:1905] T a144caea5d504a858e70809f04daddbb P 16939728527c4f3287b355862cc6a6f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:47:19.523897 20296 ts_tablet_manager.cc:1918] T a144caea5d504a858e70809f04daddbb P 783323bb150f42aeb023d8577443ffc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:47:19.524009 20296 log.cc:1196] T a144caea5d504a858e70809f04daddbb P 783323bb150f42aeb023d8577443ffc0: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/a144caea5d504a858e70809f04daddbb
I20260811 12:47:19.524384 20296 ts_tablet_manager.cc:1939] T a144caea5d504a858e70809f04daddbb P 783323bb150f42aeb023d8577443ffc0: Deleting consensus metadata
I20260811 12:47:19.525219 18997 catalog_manager.cc:4841] TS 783323bb150f42aeb023d8577443ffc0 (127.7.111.65:33335): tablet a144caea5d504a858e70809f04daddbb (table test-table [id=0adfcbdfa71f47c2aa1e9ff32975ad4e]) successfully deleted
I20260811 12:47:19.529234 20298 ts_tablet_manager.cc:1918] T a144caea5d504a858e70809f04daddbb P 16939728527c4f3287b355862cc6a6f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:47:19.529407 20298 log.cc:1196] T a144caea5d504a858e70809f04daddbb P 16939728527c4f3287b355862cc6a6f0: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/a144caea5d504a858e70809f04daddbb
I20260811 12:47:19.529860 20298 ts_tablet_manager.cc:1939] T a144caea5d504a858e70809f04daddbb P 16939728527c4f3287b355862cc6a6f0: Deleting consensus metadata
I20260811 12:47:19.530392 20254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.512470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45622 (local address 127.7.111.65:33335)
0811 12:47:19.512653 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:19.512659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:19.513448 (+ 789us) server_negotiation.cc:407] Connection header received
0811 12:47:19.513748 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:19.513755 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:19.513842 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:19.513913 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:19.516128 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:19.517398 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:19.519314 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:19.520051 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:19.520145 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:19.523711 (+ 3566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:19.523731 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:19.523754 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:19.523799 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:19.527665 (+ 3866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:19.528299 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:19.528310 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:19.528313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:19.528467 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:19.528755 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:19.528760 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:19.528763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:19.529212 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:19.529543 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:19.529819 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 12:47:19.530045 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 12:47:19.530903 18995 catalog_manager.cc:4841] TS 16939728527c4f3287b355862cc6a6f0 (127.7.111.66:46081): tablet a144caea5d504a858e70809f04daddbb (table test-table [id=0adfcbdfa71f47c2aa1e9ff32975ad4e]) successfully deleted
I20260811 12:47:19.531033 19609 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:19.623911 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 19295
I20260811 12:47:19.651953 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 19632
I20260811 12:47:19.664816 20260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.652691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33834 (local address 127.7.111.66:46081)
0811 12:47:19.652874 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:19.652878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:19.652944 (+ 66us) server_negotiation.cc:407] Connection header received
0811 12:47:19.653110 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:19.653114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:19.653194 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:19.653289 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:19.654667 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:19.655401 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:19.656681 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:19.657101 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:19.657226 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:19.659842 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:19.659859 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:19.659877 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:19.659912 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:19.662544 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:19.663146 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:19.663154 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:19.663156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:19.663296 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:19.663567 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:19.663569 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:19.663572 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:19.663907 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:19.664160 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:19.664323 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 12:47:19.664490 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 12:47:19.665689 19841 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:19.720595 20308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.719886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.0.0.1:57486)
0811 12:47:19.720331 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.720424 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":221,"threads_started":1}
I20260811 12:47:19.721971 20308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.721621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34613 (local address 127.0.0.1:57496)
0811 12:47:19.721772 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:19.721855 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 12:47:19.765856 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 19632
I20260811 12:47:19.788144 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 19872
I20260811 12:47:19.800705 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:19.788977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41276 (local address 127.7.111.67:39893)
0811 12:47:19.789165 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:19.789170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:19.789188 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:47:19.789354 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:19.789358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:19.789494 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:19.789588 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:19.790908 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:19.791614 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:19.792737 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:19.793138 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:19.793239 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:19.795743 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:19.795758 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:19.795776 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:19.795806 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:19.798412 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:19.799011 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:19.799019 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:19.799021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:19.799143 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:19.799429 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:19.799431 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:19.799433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:19.799776 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:19.800008 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:19.800218 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 12:47:19.800387 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:47:19.801344 20088 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:19.926182 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 19872
I20260811 12:47:19.957650 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 18793
I20260811 12:47:19.958782 19072 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:20.084306 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 18793
I20260811 12:47:20.112864 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 18963
I20260811 12:47:20.113967 19380 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:20.231643 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 18963
2026-08-11T12:47:20Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (12342 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:47:20 dist-test-slave-f0tn krb5kdc[20317](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:47:20 dist-test-slave-f0tn krb5kdc[20317](info): set up 2 sockets
Aug 11 12:47:20 dist-test-slave-f0tn krb5kdc[20317](info): commencing operation
krb5kdc: starting...
W20260811 12:47:21.480444 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:47:21 dist-test-slave-f0tn krb5kdc[20317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:47:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:47:21Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.126.keytab.
Entry for principal oryx/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.126.keytab.
I20260811 12:47:21.762442 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:36621
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.126.keytab
--principal=oryx/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:36621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:47:22.096127 20339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:22.096472 20339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:22.096611 20339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:22.103935 20339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:47:22.104084 20339 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 12:47:22.104161 20339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:22.104251 20339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:22.104321 20339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:47:22.104405 20339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:47:22.114230 20339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:36621
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.126.keytab
--principal=oryx/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:36621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:22.115535 20339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:22.116885 20339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:22.128482 20344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:22.128492 20348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:22.128440 20345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:22.129266 20339 server_base.cc:1034] running on GCE node
I20260811 12:47:22.130043 20339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:22.131340 20339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:22.132560 20339 hybrid_clock.cc:648] HybridClock initialized: now 1786452442132492 us; error 37 us; skew 500 ppm
I20260811 12:47:22.132889 20339 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:22 dist-test-slave-f0tn krb5kdc[20317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452442, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:22.135893 20339 init.cc:377] Logged in from keytab as oryx/127.7.111.126@KRBTEST.COM (short username oryx)
I20260811 12:47:22.137331 20339 webserver.cc:460] Webserver started at http://127.7.111.126:39115/ using document root <none> and password file <none>
I20260811 12:47:22.138025 20339 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:22.138126 20339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:22.138456 20339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:22.141352 20339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "9079ec9fcd5047f085fc64150f49664c"
format_stamp: "Formatted at 2026-08-11 12:47:22 on dist-test-slave-f0tn"
server_key: "e1677a0a9e62b406d9d39fcafd7b9f6f"
server_key_iv: "1b5070ac9df0216224e9b3f6e961cdcf"
server_key_version: "encryptionkey@0"
I20260811 12:47:22.142185 20339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "9079ec9fcd5047f085fc64150f49664c"
format_stamp: "Formatted at 2026-08-11 12:47:22 on dist-test-slave-f0tn"
server_key: "e1677a0a9e62b406d9d39fcafd7b9f6f"
server_key_iv: "1b5070ac9df0216224e9b3f6e961cdcf"
server_key_version: "encryptionkey@0"
I20260811 12:47:22.147258 20339 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 12:47:22.150635 20363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:22.151906 20339 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:47:22.152066 20339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "9079ec9fcd5047f085fc64150f49664c"
format_stamp: "Formatted at 2026-08-11 12:47:22 on dist-test-slave-f0tn"
server_key: "e1677a0a9e62b406d9d39fcafd7b9f6f"
server_key_iv: "1b5070ac9df0216224e9b3f6e961cdcf"
server_key_version: "encryptionkey@0"
I20260811 12:47:22.152271 20339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:22.175145 20339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:22.179170 20339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:22.179419 20339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:22.199471 20339 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:36621
I20260811 12:47:22.199502 20452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:36621 every 8 connection(s)
I20260811 12:47:22.200788 20339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:47:22.204506 20454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:22.208874 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 20339
I20260811 12:47:22.209092 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:47:22.209928 7613 external_mini_cluster.cc:1428] Setting key cb4d5020b4489e2cf3f9b5e0d751b545
I20260811 12:47:22.216318 20454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c: Bootstrap starting.
I20260811 12:47:22.219928 20454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:22.221050 20454 log.cc:824] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c: Log is configured to *not* fsync() on all Append() calls
I20260811 12:47:22.224440 20454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c: No bootstrap required, opened a new log
Aug 11 12:47:22 dist-test-slave-f0tn krb5kdc[20317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM
I20260811 12:47:22.234690 20461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:22.212072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41886 (local address 127.7.111.126:36621)
0811 12:47:22.212892 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:22.212904 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:22.212949 (+ 45us) server_negotiation.cc:407] Connection header received
0811 12:47:22.214772 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:22.214796 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:22.215206 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:22.215520 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:22.217262 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:22.218631 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:22.220305 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:22.220978 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:22.221120 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:22.225785 (+ 4665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:22.225796 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:22.225817 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:22.225864 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:22.230000 (+ 4136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:22.230692 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:22.230701 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:22.230710 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:22.230921 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:22.231275 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:22.231283 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:22.231287 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:22.232037 (+ 750us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:22.232326 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:22.232773 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 12:47:22.233241 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":364,"threads_started":1}
I20260811 12:47:22.235126 20454 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9079ec9fcd5047f085fc64150f49664c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36621 } }
I20260811 12:47:22.235486 20454 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:22.235584 20454 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9079ec9fcd5047f085fc64150f49664c, State: Initialized, Role: FOLLOWER
I20260811 12:47:22.236151 20454 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9079ec9fcd5047f085fc64150f49664c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36621 } }
I20260811 12:47:22.236398 20454 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:47:22.236536 20454 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:47:22.236703 20454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:22.238943 20454 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9079ec9fcd5047f085fc64150f49664c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36621 } }
I20260811 12:47:22.239475 20454 leader_election.cc:302] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9079ec9fcd5047f085fc64150f49664c; no voters:
I20260811 12:47:22.240043 20454 leader_election.cc:288] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:47:22.240154 20467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:47:22.240861 20467 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c [term 1 LEADER]: Becoming Leader. State: Replica: 9079ec9fcd5047f085fc64150f49664c, State: Running, Role: LEADER
I20260811 12:47:22.241422 20467 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9079ec9fcd5047f085fc64150f49664c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36621 } }
I20260811 12:47:22.242051 20454 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:47:22.244100 20471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9079ec9fcd5047f085fc64150f49664c. Latest consensus state: current_term: 1 leader_uuid: "9079ec9fcd5047f085fc64150f49664c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9079ec9fcd5047f085fc64150f49664c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36621 } } }
I20260811 12:47:22.244513 20471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:22.245059 20469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9079ec9fcd5047f085fc64150f49664c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9079ec9fcd5047f085fc64150f49664c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36621 } } }
I20260811 12:47:22.245394 20469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:22.245409 20484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:47:22.251744 20484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:47:22.260152 20484 catalog_manager.cc:1347] Generated new cluster ID: b2feab00d07844a7a3c34e0f7358eb06
I20260811 12:47:22.260269 20484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:47:22.273939 20484 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:47:22.275264 20484 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:47:22.285591 20484 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c: Generated new TSK 0
I20260811 12:47:22.286379 20484 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.111.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.111.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.111.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.65.keytab.
Entry for principal oryx/127.7.111.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.65.keytab.
I20260811 12:47:22.358937 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:36621
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.65.keytab
--principal=oryx/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:47:22.729252 20509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:22.729584 20509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:22.729732 20509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:22.737274 20509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:22.737412 20509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:22.737545 20509 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:47:22.737654 20509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:47:22.748093 20509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.65.keytab
--principal=oryx/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.111.126:36621
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:22.749518 20509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:22.750921 20509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:22.763863 20519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:22.763890 20518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:22.763911 20521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:22.764752 20509 server_base.cc:1034] running on GCE node
I20260811 12:47:22.765439 20509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:22.766351 20509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:22.767555 20509 hybrid_clock.cc:648] HybridClock initialized: now 1786452442767479 us; error 57 us; skew 500 ppm
I20260811 12:47:22.767843 20509 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:22 dist-test-slave-f0tn krb5kdc[20317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452442, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:22.770916 20509 init.cc:377] Logged in from keytab as oryx/127.7.111.65@KRBTEST.COM (short username oryx)
I20260811 12:47:22.772346 20509 webserver.cc:460] Webserver started at http://127.7.111.65:34921/ using document root <none> and password file <none>
I20260811 12:47:22.773023 20509 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:22.773125 20509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:22.773494 20509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:22.776523 20509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "3a1a6c3187ba42caac4b7a4a6a30933d"
format_stamp: "Formatted at 2026-08-11 12:47:22 on dist-test-slave-f0tn"
server_key: "de2b71e0c57eb33096d58680eba44cb7"
server_key_iv: "3efd7ca4c6750a91c422d2f6a5ff7346"
server_key_version: "encryptionkey@0"
I20260811 12:47:22.777309 20509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "3a1a6c3187ba42caac4b7a4a6a30933d"
format_stamp: "Formatted at 2026-08-11 12:47:22 on dist-test-slave-f0tn"
server_key: "de2b71e0c57eb33096d58680eba44cb7"
server_key_iv: "3efd7ca4c6750a91c422d2f6a5ff7346"
server_key_version: "encryptionkey@0"
I20260811 12:47:22.782521 20509 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 12:47:22.786000 20532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:22.787262 20509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:47:22.787418 20509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "3a1a6c3187ba42caac4b7a4a6a30933d"
format_stamp: "Formatted at 2026-08-11 12:47:22 on dist-test-slave-f0tn"
server_key: "de2b71e0c57eb33096d58680eba44cb7"
server_key_iv: "3efd7ca4c6750a91c422d2f6a5ff7346"
server_key_version: "encryptionkey@0"
I20260811 12:47:22.787642 20509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:22.804214 20509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:22.808224 20509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:22.808495 20509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:22.811332 20509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:22.811424 20509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:22.811621 20509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:22.811730 20509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:47:22 dist-test-slave-f0tn krb5kdc[20317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452442, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM
I20260811 12:47:22.835189 20549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:22.811742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:34581 (local address 127.7.111.126:36621)
0811 12:47:22.812258 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:22.812265 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:22.813420 (+ 1155us) server_negotiation.cc:407] Connection header received
0811 12:47:22.815541 (+ 2121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:22.815548 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:22.815636 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:22.815744 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:22.817852 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:22.818937 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:22.821912 (+ 2975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:22.822419 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:22.822557 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:22.827585 (+ 5028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:22.827591 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:22.827594 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:22.827629 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:22.831035 (+ 3406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:22.832093 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:22.832100 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:22.832104 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:22.832275 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:22.832997 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:22.833003 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:22.833007 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:22.833398 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:22.833575 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:22.834481 (+ 906us) server_negotiation.cc:299] Negotiation successful
0811 12:47:22.834744 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":246,"threads_started":1}
I20260811 12:47:22.836428 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:22.812284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:36621 (local address 127.7.111.65:34581)
0811 12:47:22.813230 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:22.813276 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:22.815229 (+ 1953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:22.816009 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:22.816018 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:22.816764 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:22.817664 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:22.817694 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:22.819126 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:22.819133 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:22.821703 (+ 2570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:22.821735 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:22.822819 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:22.822833 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:22.823232 (+ 399us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:22.824014 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:22.824056 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:22.827344 (+ 3288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:22.831201 (+ 3857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:22.831215 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:22.831230 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:22.831924 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:22.832439 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:22.832445 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:22.832448 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:22.832787 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:22.833650 (+ 863us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:22.833656 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:22.834041 (+ 385us) client_negotiation.cc:769] Sending connection context
0811 12:47:22.834316 (+ 275us) client_negotiation.cc:240] Negotiation successful
0811 12:47:22.834890 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":356,"threads_started":1}
I20260811 12:47:22.856649 20509 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:41091
I20260811 12:47:22.856675 20731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:41091 every 8 connection(s)
I20260811 12:47:22.858246 20509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:47:22.859409 20549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:22.843942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:55091 (local address 127.7.111.126:36621)
0811 12:47:22.844199 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:22.844206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:22.845160 (+ 954us) server_negotiation.cc:407] Connection header received
0811 12:47:22.845352 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:22.845357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:22.845473 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:22.845569 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:22.846805 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:22.847806 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:22.849944 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:22.850487 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:22.850610 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:22.852997 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:22.853005 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:22.853011 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:22.853057 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:22.856496 (+ 3439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:22.857262 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:22.857267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:22.857270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:22.857474 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:22.857830 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:22.857835 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:22.857838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:22.858203 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:22.858375 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:22.858800 (+ 425us) server_negotiation.cc:299] Negotiation successful
0811 12:47:22.859040 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260811 12:47:22.859476 20693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:22.844357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:36621 (local address 127.7.111.65:55091)
0811 12:47:22.845029 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:22.845057 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:22.845221 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:22.845646 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:22.845652 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:22.846259 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:22.846640 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:22.846653 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:22.847974 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:22.847980 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:22.849776 (+ 1796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:22.849798 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:22.850848 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:22.850855 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:22.851040 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:22.851709 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:22.851734 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:22.852775 (+ 1041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:22.856642 (+ 3867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:22.856648 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:22.856652 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:22.857141 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:22.857589 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:22.857592 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:22.857594 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:22.857720 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:22.858340 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:22.858345 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:22.858522 (+ 177us) client_negotiation.cc:769] Sending connection context
0811 12:47:22.858637 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 12:47:22.858989 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":393,"threads_started":1}
I20260811 12:47:22.860410 20733 heartbeater.cc:347] Connected to a master server at 127.7.111.126:36621
I20260811 12:47:22.860690 20733 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:22.861732 20733 heartbeater.cc:511] Master 127.7.111.126:36621 requested a full tablet report, sending...
I20260811 12:47:22.864217 20391 ts_manager.cc:194] Registered new tserver with Master: 3a1a6c3187ba42caac4b7a4a6a30933d (127.7.111.65:41091)
I20260811 12:47:22.864344 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 20509
I20260811 12:47:22.864543 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:47:22.865170 7613 external_mini_cluster.cc:1428] Setting key f4015bcaef54991abcffacaac18e669d
I20260811 12:47:22.865854 20391 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:34581
WARNING: no policy specified for oryx/127.7.111.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.111.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.111.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.66.keytab.
Entry for principal oryx/127.7.111.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.66.keytab.
I20260811 12:47:22.922163 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:36621
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.66.keytab
--principal=oryx/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:47:23.248267 20738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:23.248543 20738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:23.248647 20738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:23.255709 20738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:23.255805 20738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:23.255910 20738 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:47:23.255996 20738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:47:23.265714 20738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.66.keytab
--principal=oryx/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.111.126:36621
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:23.266960 20738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:23.268316 20738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:23.281368 20755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:23.281452 20752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:23.281476 20753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:23.282507 20738 server_base.cc:1034] running on GCE node
I20260811 12:47:23.283126 20738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:23.284073 20738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:23.285295 20738 hybrid_clock.cc:648] HybridClock initialized: now 1786452443285227 us; error 38 us; skew 500 ppm
I20260811 12:47:23.285619 20738 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:23 dist-test-slave-f0tn krb5kdc[20317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452443, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:23.288749 20738 init.cc:377] Logged in from keytab as oryx/127.7.111.66@KRBTEST.COM (short username oryx)
I20260811 12:47:23.290232 20738 webserver.cc:460] Webserver started at http://127.7.111.66:40133/ using document root <none> and password file <none>
I20260811 12:47:23.290987 20738 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:23.291113 20738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:23.291455 20738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:23.294680 20738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "33f78dce92df4e1eb43f72d8e07ab67b"
format_stamp: "Formatted at 2026-08-11 12:47:23 on dist-test-slave-f0tn"
server_key: "f8ff49fdf80063c0605b707999317a48"
server_key_iv: "5826fa11be0d0652549897b351cd6571"
server_key_version: "encryptionkey@0"
I20260811 12:47:23.295456 20738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "33f78dce92df4e1eb43f72d8e07ab67b"
format_stamp: "Formatted at 2026-08-11 12:47:23 on dist-test-slave-f0tn"
server_key: "f8ff49fdf80063c0605b707999317a48"
server_key_iv: "5826fa11be0d0652549897b351cd6571"
server_key_version: "encryptionkey@0"
I20260811 12:47:23.300730 20738 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:47:23.304425 20778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:23.305876 20738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:47:23.306015 20738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "33f78dce92df4e1eb43f72d8e07ab67b"
format_stamp: "Formatted at 2026-08-11 12:47:23 on dist-test-slave-f0tn"
server_key: "f8ff49fdf80063c0605b707999317a48"
server_key_iv: "5826fa11be0d0652549897b351cd6571"
server_key_version: "encryptionkey@0"
I20260811 12:47:23.306207 20738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:23.323580 20738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:23.327811 20738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:23.328037 20738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:23.330844 20738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:23.330935 20738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:23.331094 20738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:23.331156 20738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:47:23 dist-test-slave-f0tn krb5kdc[20317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452443, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM
I20260811 12:47:23.353652 20549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:23.331252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:51013 (local address 127.7.111.126:36621)
0811 12:47:23.331503 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:23.331510 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:23.332911 (+ 1401us) server_negotiation.cc:407] Connection header received
0811 12:47:23.335018 (+ 2107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:23.335025 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:23.335119 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:23.335276 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:23.337352 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:23.338366 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:23.340771 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:23.341345 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:23.341500 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:23.346584 (+ 5084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:23.346591 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:23.346597 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:23.346643 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:23.349897 (+ 3254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:23.350856 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:23.350862 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:23.350866 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:23.350998 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:23.351571 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:23.351577 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:23.351580 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:23.351956 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:23.352111 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:23.353044 (+ 933us) server_negotiation.cc:299] Negotiation successful
0811 12:47:23.353270 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 12:47:23.355383 20796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:23.331841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:36621 (local address 127.7.111.66:51013)
0811 12:47:23.332727 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:23.332776 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:23.334723 (+ 1947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:23.335546 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:23.335556 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:23.336253 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:23.337172 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:23.337198 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:23.338544 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:23.338550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:23.340639 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:23.340651 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:23.341636 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:23.341645 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:23.342042 (+ 397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:23.342873 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:23.342907 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:23.346303 (+ 3396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:23.350059 (+ 3756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:23.350068 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:23.350086 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:23.350730 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:23.351132 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:23.351141 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:23.351144 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:23.351458 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:23.352098 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:23.352106 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:23.352560 (+ 454us) client_negotiation.cc:769] Sending connection context
0811 12:47:23.352908 (+ 348us) client_negotiation.cc:240] Negotiation successful
0811 12:47:23.353547 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":376,"threads_started":1}
I20260811 12:47:23.375294 20970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:23.362444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:36621 (local address 127.7.111.66:35579)
0811 12:47:23.363116 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:23.363142 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:23.363351 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:23.363718 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:23.363723 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:23.364377 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:23.364682 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:23.364692 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:23.365747 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:23.365753 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:23.366805 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:23.366818 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:23.367533 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:23.367540 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:23.367690 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:23.368353 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:23.368370 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:23.369098 (+ 728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:23.372419 (+ 3321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:23.372425 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:23.372429 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:23.372924 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:23.373436 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:23.373440 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:23.373442 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:23.373565 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:23.374295 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:23.374305 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:23.374499 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 12:47:23.374627 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 12:47:23.374899 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":359,"threads_started":1}
I20260811 12:47:23.375415 20549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:23.362333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:35579 (local address 127.7.111.126:36621)
0811 12:47:23.362787 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:23.362793 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:23.363230 (+ 437us) server_negotiation.cc:407] Connection header received
0811 12:47:23.363516 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:23.363521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:23.363591 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:23.363671 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:23.364821 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:23.365617 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:23.366938 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:23.367345 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:23.367452 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:23.369273 (+ 1821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:23.369278 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:23.369281 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:23.369320 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:23.372283 (+ 2963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:23.373114 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:23.373122 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:23.373126 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:23.373306 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:23.373664 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:23.373669 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:23.373673 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:23.374142 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:23.374297 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:23.374755 (+ 458us) server_negotiation.cc:299] Negotiation successful
0811 12:47:23.375035 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312}
I20260811 12:47:23.378865 20738 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:46079
I20260811 12:47:23.378913 21006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:46079 every 8 connection(s)
I20260811 12:47:23.380100 20738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:47:23.382105 21008 heartbeater.cc:347] Connected to a master server at 127.7.111.126:36621
I20260811 12:47:23.382316 21008 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:23.382947 21008 heartbeater.cc:511] Master 127.7.111.126:36621 requested a full tablet report, sending...
I20260811 12:47:23.384152 20389 ts_manager.cc:194] Registered new tserver with Master: 33f78dce92df4e1eb43f72d8e07ab67b (127.7.111.66:46079)
I20260811 12:47:23.385006 20389 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:51013
I20260811 12:47:23.387217 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 20738
I20260811 12:47:23.387372 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:47:23.387794 7613 external_mini_cluster.cc:1428] Setting key d2d563d7d22a49ea4a715a53b31b5062
WARNING: no policy specified for oryx/127.7.111.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.111.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.111.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.67.keytab.
Entry for principal oryx/127.7.111.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.67.keytab.
I20260811 12:47:23.442258 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:36621
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.67.keytab
--principal=oryx/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:47:23.776230 21015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:23.776520 21015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:23.776666 21015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:23.783991 21015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:23.784114 21015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:23.784240 21015 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:47:23.784351 21015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:47:23.794181 21015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.67.keytab
--principal=oryx/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.111.126:36621
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:23.795545 21015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:23.796941 21015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:23.809934 21031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:23.809998 21028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:23.810019 21027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:23.811100 21015 server_base.cc:1034] running on GCE node
I20260811 12:47:23.811734 21015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:23.812690 21015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:23.813913 21015 hybrid_clock.cc:648] HybridClock initialized: now 1786452443813830 us; error 58 us; skew 500 ppm
I20260811 12:47:23.814225 21015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:23 dist-test-slave-f0tn krb5kdc[20317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452443, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:23.817289 21015 init.cc:377] Logged in from keytab as oryx/127.7.111.67@KRBTEST.COM (short username oryx)
I20260811 12:47:23.818783 21015 webserver.cc:460] Webserver started at http://127.7.111.67:40055/ using document root <none> and password file <none>
I20260811 12:47:23.819485 21015 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:23.819586 21015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:23.819901 21015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:23.822889 21015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "fca017e7ad634088988f027f77db0b8f"
format_stamp: "Formatted at 2026-08-11 12:47:23 on dist-test-slave-f0tn"
server_key: "908036e8ee1e33c9191dbbbd6305de09"
server_key_iv: "e5c290ca828aa16666fea1fa9f4c9c86"
server_key_version: "encryptionkey@0"
I20260811 12:47:23.823664 21015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "fca017e7ad634088988f027f77db0b8f"
format_stamp: "Formatted at 2026-08-11 12:47:23 on dist-test-slave-f0tn"
server_key: "908036e8ee1e33c9191dbbbd6305de09"
server_key_iv: "e5c290ca828aa16666fea1fa9f4c9c86"
server_key_version: "encryptionkey@0"
I20260811 12:47:23.828732 21015 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 12:47:23.832162 21047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:23.833504 21015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:47:23.833657 21015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "fca017e7ad634088988f027f77db0b8f"
format_stamp: "Formatted at 2026-08-11 12:47:23 on dist-test-slave-f0tn"
server_key: "908036e8ee1e33c9191dbbbd6305de09"
server_key_iv: "e5c290ca828aa16666fea1fa9f4c9c86"
server_key_version: "encryptionkey@0"
I20260811 12:47:23.833869 21015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:23.846724 21015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:23.850628 21015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:23.850878 21015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:23.853576 21015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:23.853662 21015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:23.853799 21015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:23.853857 21015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:47:23 dist-test-slave-f0tn krb5kdc[20317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452443, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM
I20260811 12:47:23.869601 20733 heartbeater.cc:503] Master 127.7.111.126:36621 was elected leader, sending a full tablet report...
I20260811 12:47:23.875409 20549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:23.854204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:49861 (local address 127.7.111.126:36621)
0811 12:47:23.854405 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:23.854411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:23.855719 (+ 1308us) server_negotiation.cc:407] Connection header received
0811 12:47:23.858070 (+ 2351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:23.858075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:23.858169 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:23.858268 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:23.860594 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:23.861709 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:23.863731 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:23.864154 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:23.864269 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:23.868806 (+ 4537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:23.868813 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:23.868817 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:23.868861 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:23.871950 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:23.872887 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:23.872892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:23.872894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:23.873059 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:23.873556 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:23.873562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:23.873565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:23.873889 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:23.874071 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:23.874805 (+ 734us) server_negotiation.cc:299] Negotiation successful
0811 12:47:23.875069 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:47:23.876464 21072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:23.854690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:36621 (local address 127.7.111.67:49861)
0811 12:47:23.855554 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:23.855604 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:23.857809 (+ 2205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:23.858730 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:23.858740 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:23.859442 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:23.860339 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:23.860369 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:23.861915 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:23.861921 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:23.863585 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:23.863598 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:23.864383 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:23.864391 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:23.864676 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:23.865367 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:23.865417 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:23.868489 (+ 3072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:23.872105 (+ 3616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:23.872115 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:23.872132 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:23.872751 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:23.873156 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:23.873161 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:23.873165 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:23.873439 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:23.874043 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:23.874051 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:23.874375 (+ 324us) client_negotiation.cc:769] Sending connection context
0811 12:47:23.874622 (+ 247us) client_negotiation.cc:240] Negotiation successful
0811 12:47:23.875120 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":393,"threads_started":1}
I20260811 12:47:23.896574 20549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:23.882938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:44957 (local address 127.7.111.126:36621)
0811 12:47:23.883142 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:23.883148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:23.883691 (+ 543us) server_negotiation.cc:407] Connection header received
0811 12:47:23.883874 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:23.883879 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:23.883962 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:23.884069 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:23.884898 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:23.886001 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:23.887664 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:23.888200 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:23.888319 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:23.890891 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:23.890899 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:23.890903 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:23.890943 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:23.893712 (+ 2769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:23.894666 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:23.894672 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:23.894676 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:23.894802 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:23.895156 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:23.895162 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:23.895165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:23.895431 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:23.895591 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:23.895987 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 12:47:23.896204 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:47:23.896658 21162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:23.882955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:36621 (local address 127.7.111.67:44957)
0811 12:47:23.883586 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:23.883610 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:23.883765 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:23.884080 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:23.884083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:23.884483 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:23.884780 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:23.884789 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:23.886158 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:23.886164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:23.887525 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:23.887540 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:23.888586 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:23.888593 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:23.888883 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:23.889567 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:23.889592 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:23.890709 (+ 1117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:23.893873 (+ 3164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:23.893879 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:23.893883 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:23.894555 (+ 672us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:23.894908 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:23.894914 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:23.894918 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:23.895055 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:23.895548 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:23.895553 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:23.895736 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 12:47:23.895891 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 12:47:23.896235 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":331,"threads_started":1}
I20260811 12:47:23.907186 21015 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:43503
I20260811 12:47:23.907222 21254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:43503 every 8 connection(s)
I20260811 12:47:23.908411 21015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:47:23.910455 21255 heartbeater.cc:347] Connected to a master server at 127.7.111.126:36621
I20260811 12:47:23.910697 21255 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:23.911371 21255 heartbeater.cc:511] Master 127.7.111.126:36621 requested a full tablet report, sending...
I20260811 12:47:23.912669 20389 ts_manager.cc:194] Registered new tserver with Master: fca017e7ad634088988f027f77db0b8f (127.7.111.67:43503)
I20260811 12:47:23.913523 20389 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:49861
I20260811 12:47:23.916568 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 21015
I20260811 12:47:23.916731 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:47:23.917145 7613 external_mini_cluster.cc:1428] Setting key baaa1cc2c43419e333379197492ff423
I20260811 12:47:23.919813 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 12:47:23 dist-test-slave-f0tn krb5kdc[20317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452443, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 12:47:23 dist-test-slave-f0tn krb5kdc[20317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:23 dist-test-slave-f0tn krb5kdc[20317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:24.387288 21008 heartbeater.cc:503] Master 127.7.111.126:36621 was elected leader, sending a full tablet report...
I20260811 12:47:24.915861 21255 heartbeater.cc:503] Master 127.7.111.126:36621 was elected leader, sending a full tablet report...
W20260811 12:47:25.033808 21265 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.061s user 0.005s sys 0.014s
W20260811 12:47:25.035076 20549 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:23.967699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41890 (local address 127.7.111.126:36621)
0811 12:47:23.967866 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:23.967870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:23.968141 (+ 271us) server_negotiation.cc:407] Connection header received
0811 12:47:23.968294 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:23.968298 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:23.968364 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:23.968451 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:23.969770 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:23.970510 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:23.971671 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:23.972093 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:23.972185 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:25.034757 (+1062572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41890: BlockingRecv error: recv got EOF from 127.0.0.1:41890 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 12:47:25 dist-test-slave-f0tn krb5kdc[20317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452443, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM
I20260811 12:47:25.059513 20549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:25.039274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52764 (local address 127.7.111.126:36621)
0811 12:47:25.039466 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:25.039473 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:25.039808 (+ 335us) server_negotiation.cc:407] Connection header received
0811 12:47:25.039997 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:25.040001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:25.040084 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:25.040173 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:25.041530 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.042362 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.043706 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.044132 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.044249 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:25.054368 (+ 10119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:25.054374 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:25.054378 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:25.054424 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:25.057096 (+ 2672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:25.057816 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:25.057821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:25.057825 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:25.057963 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:25.058403 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:25.058407 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:25.058409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:25.058700 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:25.058926 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:25.059106 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 12:47:25.059292 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 12:47:25.063558 20389 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52764:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:47:25.070539 20389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:47:25.097523 21291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:25.083589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33280 (local address 127.7.111.67:43503)
0811 12:47:25.088753 (+ 5164us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:25.088764 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:25.089369 (+ 605us) server_negotiation.cc:407] Connection header received
0811 12:47:25.089715 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:25.089723 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:25.089945 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:25.090078 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:47:25.091444 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.093064 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.095230 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.096404 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.096528 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:25.096748 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:25.097038 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 12:47:25.097214 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4972,"thread_start_us":4813,"threads_started":1}
I20260811 12:47:25.097604 21285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:25.083041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:43503 (local address 127.0.0.1:33280)
0811 12:47:25.089238 (+ 6197us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:25.089259 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:25.089514 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:25.090323 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:25.090331 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:25.090370 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:47:25.091190 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:25.091205 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.093290 (+ 2085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.093295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:25.095023 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:25.095039 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.096601 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.096607 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:25.096806 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:25.096849 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 12:47:25.096967 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 12:47:25.097166 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6041,"mutex_wait_us":163,"thread_start_us":720,"threads_started":1}
I20260811 12:47:25.098340 21289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:25.082672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:46079 (local address 127.0.0.1:33648)
0811 12:47:25.083786 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:25.083832 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:25.084034 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:25.090359 (+ 6325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:25.090363 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:25.090381 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:47:25.091322 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:25.091340 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.092865 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.092870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:25.095217 (+ 2347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:25.095233 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.097403 (+ 2170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.097411 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:25.097572 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:25.097606 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 12:47:25.097775 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 12:47:25.097899 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"mutex_wait_us":151,"spinlock_wait_cycles":4864,"thread_start_us":6264,"threads_started":1}
I20260811 12:47:25.098819 21288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:25.082908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33648 (local address 127.7.111.66:46079)
0811 12:47:25.089838 (+ 6930us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:25.089846 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:25.089868 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:47:25.089947 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:25.089953 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:25.090168 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:25.090363 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:47:25.091538 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.092744 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.095700 (+ 2956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.097098 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.098109 (+ 1011us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:25.098304 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:25.098371 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 12:47:25.098528 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6761,"thread_start_us":6628,"threads_started":1}
I20260811 12:47:25.099107 21290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:25.083295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:41091 (local address 127.0.0.1:36538)
0811 12:47:25.089555 (+ 6260us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:25.089572 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:25.089696 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:25.090633 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:25.090637 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:25.090657 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:47:25.091470 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:25.091484 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.093476 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.093481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:25.095676 (+ 2195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:25.095689 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.098052 (+ 2363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.098060 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:25.098229 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:25.098259 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 12:47:25.098362 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 12:47:25.098758 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6141,"mutex_wait_us":24,"thread_start_us":5960,"threads_started":1}
I20260811 12:47:25.099920 21293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:25.083861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36538 (local address 127.7.111.65:41091)
0811 12:47:25.089072 (+ 5211us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:25.089080 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:25.089826 (+ 746us) server_negotiation.cc:407] Connection header received
0811 12:47:25.089932 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:25.089939 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:25.090168 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:25.090363 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:47:25.091808 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.093271 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.095861 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.097198 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.097319 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:25.097551 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:25.099452 (+ 1901us) server_negotiation.cc:299] Negotiation successful
0811 12:47:25.099630 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5064,"thread_start_us":4922,"threads_started":1}
I20260811 12:47:25.102159 20862 tablet_service.cc:1467] Processing CreateTablet for tablet 389510e9ab5c419c9fcb4734baec688a (DEFAULT_TABLE table=test-table [id=f10f2ac0edd74ef1ad2ddd466c1369bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:47:25.102056 21141 tablet_service.cc:1467] Processing CreateTablet for tablet 389510e9ab5c419c9fcb4734baec688a (DEFAULT_TABLE table=test-table [id=f10f2ac0edd74ef1ad2ddd466c1369bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:47:25.103423 20862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 389510e9ab5c419c9fcb4734baec688a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:25.103588 21141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 389510e9ab5c419c9fcb4734baec688a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:25.103154 20617 tablet_service.cc:1467] Processing CreateTablet for tablet 389510e9ab5c419c9fcb4734baec688a (DEFAULT_TABLE table=test-table [id=f10f2ac0edd74ef1ad2ddd466c1369bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:47:25.104383 20617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 389510e9ab5c419c9fcb4734baec688a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:25.115393 21294 tablet_bootstrap.cc:492] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b: Bootstrap starting.
I20260811 12:47:25.118078 21295 tablet_bootstrap.cc:492] T 389510e9ab5c419c9fcb4734baec688a P fca017e7ad634088988f027f77db0b8f: Bootstrap starting.
I20260811 12:47:25.119053 21294 tablet_bootstrap.cc:654] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:25.120414 21294 log.cc:824] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b: Log is configured to *not* fsync() on all Append() calls
I20260811 12:47:25.121795 21295 tablet_bootstrap.cc:654] T 389510e9ab5c419c9fcb4734baec688a P fca017e7ad634088988f027f77db0b8f: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:25.121910 21296 tablet_bootstrap.cc:492] T 389510e9ab5c419c9fcb4734baec688a P 3a1a6c3187ba42caac4b7a4a6a30933d: Bootstrap starting.
I20260811 12:47:25.123127 21295 log.cc:824] T 389510e9ab5c419c9fcb4734baec688a P fca017e7ad634088988f027f77db0b8f: Log is configured to *not* fsync() on all Append() calls
I20260811 12:47:25.124375 21294 tablet_bootstrap.cc:492] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b: No bootstrap required, opened a new log
I20260811 12:47:25.124681 21294 ts_tablet_manager.cc:1397] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 12:47:25.125726 21296 tablet_bootstrap.cc:654] T 389510e9ab5c419c9fcb4734baec688a P 3a1a6c3187ba42caac4b7a4a6a30933d: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:25.126434 21295 tablet_bootstrap.cc:492] T 389510e9ab5c419c9fcb4734baec688a P fca017e7ad634088988f027f77db0b8f: No bootstrap required, opened a new log
I20260811 12:47:25.126716 21295 ts_tablet_manager.cc:1397] T 389510e9ab5c419c9fcb4734baec688a P fca017e7ad634088988f027f77db0b8f: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 12:47:25.127024 21296 log.cc:824] T 389510e9ab5c419c9fcb4734baec688a P 3a1a6c3187ba42caac4b7a4a6a30933d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:47:25.130265 21296 tablet_bootstrap.cc:492] T 389510e9ab5c419c9fcb4734baec688a P 3a1a6c3187ba42caac4b7a4a6a30933d: No bootstrap required, opened a new log
I20260811 12:47:25.130566 21296 ts_tablet_manager.cc:1397] T 389510e9ab5c419c9fcb4734baec688a P 3a1a6c3187ba42caac4b7a4a6a30933d: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 12:47:25.132465 21294 raft_consensus.cc:348] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } } peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } }
I20260811 12:47:25.132854 21294 raft_consensus.cc:374] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:25.132954 21294 raft_consensus.cc:729] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33f78dce92df4e1eb43f72d8e07ab67b, State: Initialized, Role: FOLLOWER
I20260811 12:47:25.133775 21294 consensus_queue.cc:260] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } } peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } }
I20260811 12:47:25.136189 21294 ts_tablet_manager.cc:1428] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260811 12:47:25.136991 21295 raft_consensus.cc:348] T 389510e9ab5c419c9fcb4734baec688a P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } } peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } }
I20260811 12:47:25.137558 21295 raft_consensus.cc:374] T 389510e9ab5c419c9fcb4734baec688a P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:25.137668 21295 raft_consensus.cc:729] T 389510e9ab5c419c9fcb4734baec688a P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fca017e7ad634088988f027f77db0b8f, State: Initialized, Role: FOLLOWER
I20260811 12:47:25.138366 21295 consensus_queue.cc:260] T 389510e9ab5c419c9fcb4734baec688a P fca017e7ad634088988f027f77db0b8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } } peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } }
I20260811 12:47:25.140232 21295 ts_tablet_manager.cc:1428] T 389510e9ab5c419c9fcb4734baec688a P fca017e7ad634088988f027f77db0b8f: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260811 12:47:25.140937 21296 raft_consensus.cc:348] T 389510e9ab5c419c9fcb4734baec688a P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } } peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } }
I20260811 12:47:25.141394 21296 raft_consensus.cc:374] T 389510e9ab5c419c9fcb4734baec688a P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:25.141507 21296 raft_consensus.cc:729] T 389510e9ab5c419c9fcb4734baec688a P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1a6c3187ba42caac4b7a4a6a30933d, State: Initialized, Role: FOLLOWER
I20260811 12:47:25.142264 21296 consensus_queue.cc:260] T 389510e9ab5c419c9fcb4734baec688a P 3a1a6c3187ba42caac4b7a4a6a30933d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } } peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } }
I20260811 12:47:25.144136 21296 ts_tablet_manager.cc:1428] T 389510e9ab5c419c9fcb4734baec688a P 3a1a6c3187ba42caac4b7a4a6a30933d: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
W20260811 12:47:25.160751 21256 tablet.cc:2369] T 389510e9ab5c419c9fcb4734baec688a P fca017e7ad634088988f027f77db0b8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:47:25.357352 21303 raft_consensus.cc:482] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:47:25.357793 21303 raft_consensus.cc:504] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } } peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } }
I20260811 12:47:25.359613 21303 leader_election.cc:288] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fca017e7ad634088988f027f77db0b8f (127.7.111.67:43503), 3a1a6c3187ba42caac4b7a4a6a30933d (127.7.111.65:41091)
W20260811 12:47:25.363932 20735 tablet.cc:2369] T 389510e9ab5c419c9fcb4734baec688a P 3a1a6c3187ba42caac4b7a4a6a30933d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:47:25.366811 21293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:25.360168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:39103 (local address 127.7.111.65:41091)
0811 12:47:25.360406 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:25.360413 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:25.361028 (+ 615us) server_negotiation.cc:407] Connection header received
0811 12:47:25.361211 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:25.361215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:25.361276 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:25.361369 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:47:25.362074 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.362926 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.364914 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.365863 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.365994 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:25.366059 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:25.366441 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 12:47:25.366576 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 12:47:25.366990 21291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:25.359934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:43255 (local address 127.7.111.67:43503)
0811 12:47:25.360166 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:25.360172 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:25.360956 (+ 784us) server_negotiation.cc:407] Connection header received
0811 12:47:25.361139 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:25.361145 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:25.361232 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:25.361301 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:47:25.362009 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.362751 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.365215 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.366097 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.366272 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:25.366341 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:25.366650 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 12:47:25.366779 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:47:25.367111 21318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:25.359838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:43503 (local address 127.7.111.66:43255)
0811 12:47:25.360817 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:25.360843 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:25.360986 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:25.361344 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:25.361349 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:25.361405 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:47:25.361851 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:25.361861 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.362928 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.362933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:25.365043 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:25.365059 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.366310 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.366315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:25.366483 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:25.366516 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:47:25.366609 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 12:47:25.366753 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"mutex_wait_us":75,"thread_start_us":568,"threads_started":1}
I20260811 12:47:25.367233 21321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:25.360047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:41091 (local address 127.7.111.66:39103)
0811 12:47:25.360913 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:25.360930 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:25.361067 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:25.361427 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:25.361432 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:25.361457 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:47:25.361887 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:25.361898 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.363095 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.363100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:25.364763 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:25.364778 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.366079 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.366085 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:25.366245 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:25.366284 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 12:47:25.366384 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 12:47:25.366490 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"mutex_wait_us":3,"thread_start_us":563,"threads_started":1}
I20260811 12:47:25.368089 21186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "389510e9ab5c419c9fcb4734baec688a" candidate_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fca017e7ad634088988f027f77db0b8f" is_pre_election: true
I20260811 12:47:25.368259 20653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "389510e9ab5c419c9fcb4734baec688a" candidate_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" is_pre_election: true
I20260811 12:47:25.368597 21186 raft_consensus.cc:2454] T 389510e9ab5c419c9fcb4734baec688a P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33f78dce92df4e1eb43f72d8e07ab67b in term 0.
I20260811 12:47:25.368728 20653 raft_consensus.cc:2454] T 389510e9ab5c419c9fcb4734baec688a P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33f78dce92df4e1eb43f72d8e07ab67b in term 0.
I20260811 12:47:25.369272 20779 leader_election.cc:302] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33f78dce92df4e1eb43f72d8e07ab67b, fca017e7ad634088988f027f77db0b8f; no voters:
I20260811 12:47:25.369730 21303 raft_consensus.cc:2781] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:47:25.369865 21303 raft_consensus.cc:482] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:47:25.369972 21303 raft_consensus.cc:3037] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:25.372205 21303 raft_consensus.cc:504] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } } peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } }
I20260811 12:47:25.372797 21303 leader_election.cc:288] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b [CANDIDATE]: Term 1 election: Requested vote from peers fca017e7ad634088988f027f77db0b8f (127.7.111.67:43503), 3a1a6c3187ba42caac4b7a4a6a30933d (127.7.111.65:41091)
I20260811 12:47:25.373317 21186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "389510e9ab5c419c9fcb4734baec688a" candidate_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fca017e7ad634088988f027f77db0b8f"
I20260811 12:47:25.373318 20653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "389510e9ab5c419c9fcb4734baec688a" candidate_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d"
I20260811 12:47:25.373553 20653 raft_consensus.cc:3037] T 389510e9ab5c419c9fcb4734baec688a P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:25.373553 21186 raft_consensus.cc:3037] T 389510e9ab5c419c9fcb4734baec688a P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:25.376756 21186 raft_consensus.cc:2454] T 389510e9ab5c419c9fcb4734baec688a P fca017e7ad634088988f027f77db0b8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33f78dce92df4e1eb43f72d8e07ab67b in term 1.
I20260811 12:47:25.376751 20653 raft_consensus.cc:2454] T 389510e9ab5c419c9fcb4734baec688a P 3a1a6c3187ba42caac4b7a4a6a30933d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33f78dce92df4e1eb43f72d8e07ab67b in term 1.
I20260811 12:47:25.377187 20782 leader_election.cc:302] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33f78dce92df4e1eb43f72d8e07ab67b, 3a1a6c3187ba42caac4b7a4a6a30933d; no voters:
I20260811 12:47:25.377527 21303 raft_consensus.cc:2781] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:47:25.378301 21303 raft_consensus.cc:686] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b [term 1 LEADER]: Becoming Leader. State: Replica: 33f78dce92df4e1eb43f72d8e07ab67b, State: Running, Role: LEADER
I20260811 12:47:25.378911 21303 consensus_queue.cc:237] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } } peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } }
W20260811 12:47:25.382831 21009 tablet.cc:2369] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:47:25.384053 20391 catalog_manager.cc:5495] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b reported cstate change: term changed from 0 to 1, leader changed from <none> to 33f78dce92df4e1eb43f72d8e07ab67b (127.7.111.66). New cstate: current_term: 1 leader_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } health_report { overall_health: UNKNOWN } } }
I20260811 12:47:25.461261 21288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:25.456138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33658 (local address 127.7.111.66:46079)
0811 12:47:25.456333 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:25.456337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:25.456405 (+ 68us) server_negotiation.cc:407] Connection header received
0811 12:47:25.456594 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:25.456599 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:25.456692 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:25.456818 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:47:25.457169 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.457970 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.459396 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.459800 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.459904 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:25.460260 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:47:25.460807 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:47:25.460919 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:25.460961 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 12:47:25.461034 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:47:25.469687 20653 raft_consensus.cc:1261] T 389510e9ab5c419c9fcb4734baec688a P 3a1a6c3187ba42caac4b7a4a6a30933d [term 1 FOLLOWER]: Refusing update from remote peer 33f78dce92df4e1eb43f72d8e07ab67b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:47:25.469686 21186 raft_consensus.cc:1261] T 389510e9ab5c419c9fcb4734baec688a P fca017e7ad634088988f027f77db0b8f [term 1 FOLLOWER]: Refusing update from remote peer 33f78dce92df4e1eb43f72d8e07ab67b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:47:25.470660 21303 consensus_queue.cc:1035] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:47:25.470952 21322 consensus_queue.cc:1035] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:47:25.484241 21327 mvcc.cc:205] Tried to move back new op lower bound from 7317309216633262080 to 7317309216276348928. Current Snapshot: MvccSnapshot[applied={T|T < 7317309216633262080 or (T in {7317309216633262080})}]
I20260811 12:47:25.484589 21328 mvcc.cc:205] Tried to move back new op lower bound from 7317309216633262080 to 7317309216276348928. Current Snapshot: MvccSnapshot[applied={T|T < 7317309216633262080}]
I20260811 12:47:25.494179 20389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:47:25.494488 20389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:47:25.498168 20389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c: Sending DeleteTablet for 3 replicas of tablet 389510e9ab5c419c9fcb4734baec688a
I20260811 12:47:25.499374 21141 tablet_service.cc:1514] Processing DeleteTablet for tablet 389510e9ab5c419c9fcb4734baec688a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:47:25 UTC) from {username='oryx', principal='oryx/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:33280
I20260811 12:47:25.499390 20617 tablet_service.cc:1514] Processing DeleteTablet for tablet 389510e9ab5c419c9fcb4734baec688a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:47:25 UTC) from {username='oryx', principal='oryx/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:36538
I20260811 12:47:25.499426 20862 tablet_service.cc:1514] Processing DeleteTablet for tablet 389510e9ab5c419c9fcb4734baec688a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:47:25 UTC) from {username='oryx', principal='oryx/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:33648
I20260811 12:47:25.502681 20389 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52764:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:47:25.503425 20389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:47:25.512123 21344 tablet_replica.cc:330] T 389510e9ab5c419c9fcb4734baec688a P 3a1a6c3187ba42caac4b7a4a6a30933d: stopping tablet replica
I20260811 12:47:25.512888 21344 raft_consensus.cc:2229] T 389510e9ab5c419c9fcb4734baec688a P 3a1a6c3187ba42caac4b7a4a6a30933d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:47:25.513559 21344 raft_consensus.cc:2258] T 389510e9ab5c419c9fcb4734baec688a P 3a1a6c3187ba42caac4b7a4a6a30933d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:47:25.514164 20858 tablet_service.cc:1467] Processing CreateTablet for tablet bafc890ef5dd4cbca5e81d0389051ed0 (DEFAULT_TABLE table=test-table [id=f2a8f8a4e3f44410b2593aeae1da31f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:47:25.514815 20858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bafc890ef5dd4cbca5e81d0389051ed0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:25.516078 21344 ts_tablet_manager.cc:1905] T 389510e9ab5c419c9fcb4734baec688a P 3a1a6c3187ba42caac4b7a4a6a30933d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:47:25.517213 21345 tablet_replica.cc:330] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b: stopping tablet replica
I20260811 12:47:25.517735 21345 raft_consensus.cc:2229] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:47:25.516981 20617 tablet_service.cc:1467] Processing CreateTablet for tablet bafc890ef5dd4cbca5e81d0389051ed0 (DEFAULT_TABLE table=test-table [id=f2a8f8a4e3f44410b2593aeae1da31f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:47:25.518293 20617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bafc890ef5dd4cbca5e81d0389051ed0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:25.522261 21139 tablet_service.cc:1467] Processing CreateTablet for tablet bafc890ef5dd4cbca5e81d0389051ed0 (DEFAULT_TABLE table=test-table [id=f2a8f8a4e3f44410b2593aeae1da31f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:47:25.522868 21139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bafc890ef5dd4cbca5e81d0389051ed0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:25.525624 21343 tablet_replica.cc:330] T 389510e9ab5c419c9fcb4734baec688a P fca017e7ad634088988f027f77db0b8f: stopping tablet replica
I20260811 12:47:25.526046 21343 raft_consensus.cc:2229] T 389510e9ab5c419c9fcb4734baec688a P fca017e7ad634088988f027f77db0b8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:47:25.526516 21296 tablet_bootstrap.cc:492] T bafc890ef5dd4cbca5e81d0389051ed0 P 3a1a6c3187ba42caac4b7a4a6a30933d: Bootstrap starting.
I20260811 12:47:25.526562 21343 raft_consensus.cc:2258] T 389510e9ab5c419c9fcb4734baec688a P fca017e7ad634088988f027f77db0b8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:47:25.527419 21345 raft_consensus.cc:2258] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:47:25.528277 21296 tablet_bootstrap.cc:654] T bafc890ef5dd4cbca5e81d0389051ed0 P 3a1a6c3187ba42caac4b7a4a6a30933d: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:25.529901 21343 ts_tablet_manager.cc:1905] T 389510e9ab5c419c9fcb4734baec688a P fca017e7ad634088988f027f77db0b8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:47:25.529970 21345 ts_tablet_manager.cc:1905] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:47:25.531342 21295 tablet_bootstrap.cc:492] T bafc890ef5dd4cbca5e81d0389051ed0 P fca017e7ad634088988f027f77db0b8f: Bootstrap starting.
I20260811 12:47:25.532990 21295 tablet_bootstrap.cc:654] T bafc890ef5dd4cbca5e81d0389051ed0 P fca017e7ad634088988f027f77db0b8f: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:25.537029 21343 ts_tablet_manager.cc:1918] T 389510e9ab5c419c9fcb4734baec688a P fca017e7ad634088988f027f77db0b8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:47:25.537308 21343 log.cc:1196] T 389510e9ab5c419c9fcb4734baec688a P fca017e7ad634088988f027f77db0b8f: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/389510e9ab5c419c9fcb4734baec688a
I20260811 12:47:25.537928 21343 ts_tablet_manager.cc:1939] T 389510e9ab5c419c9fcb4734baec688a P fca017e7ad634088988f027f77db0b8f: Deleting consensus metadata
I20260811 12:47:25.539579 21294 tablet_bootstrap.cc:492] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b: Bootstrap starting.
I20260811 12:47:25.540337 21345 ts_tablet_manager.cc:1918] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:47:25.540520 21345 log.cc:1196] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/389510e9ab5c419c9fcb4734baec688a
I20260811 12:47:25.541035 21345 ts_tablet_manager.cc:1939] T 389510e9ab5c419c9fcb4734baec688a P 33f78dce92df4e1eb43f72d8e07ab67b: Deleting consensus metadata
I20260811 12:47:25.541234 21294 tablet_bootstrap.cc:654] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:25.541704 20364 catalog_manager.cc:4841] TS fca017e7ad634088988f027f77db0b8f (127.7.111.67:43503): tablet 389510e9ab5c419c9fcb4734baec688a (table test-table [id=f10f2ac0edd74ef1ad2ddd466c1369bc]) successfully deleted
I20260811 12:47:25.542809 21344 ts_tablet_manager.cc:1918] T 389510e9ab5c419c9fcb4734baec688a P 3a1a6c3187ba42caac4b7a4a6a30933d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:47:25.543100 21344 log.cc:1196] T 389510e9ab5c419c9fcb4734baec688a P 3a1a6c3187ba42caac4b7a4a6a30933d: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/389510e9ab5c419c9fcb4734baec688a
I20260811 12:47:25.543723 21344 ts_tablet_manager.cc:1939] T 389510e9ab5c419c9fcb4734baec688a P 3a1a6c3187ba42caac4b7a4a6a30933d: Deleting consensus metadata
I20260811 12:47:25.545502 20367 catalog_manager.cc:4841] TS 3a1a6c3187ba42caac4b7a4a6a30933d (127.7.111.65:41091): tablet 389510e9ab5c419c9fcb4734baec688a (table test-table [id=f10f2ac0edd74ef1ad2ddd466c1369bc]) successfully deleted
I20260811 12:47:25.546674 20366 catalog_manager.cc:4841] TS 33f78dce92df4e1eb43f72d8e07ab67b (127.7.111.66:46079): tablet 389510e9ab5c419c9fcb4734baec688a (table test-table [id=f10f2ac0edd74ef1ad2ddd466c1369bc]) successfully deleted
I20260811 12:47:25.548930 21296 tablet_bootstrap.cc:492] T bafc890ef5dd4cbca5e81d0389051ed0 P 3a1a6c3187ba42caac4b7a4a6a30933d: No bootstrap required, opened a new log
I20260811 12:47:25.549082 21296 ts_tablet_manager.cc:1397] T bafc890ef5dd4cbca5e81d0389051ed0 P 3a1a6c3187ba42caac4b7a4a6a30933d: Time spent bootstrapping tablet: real 0.023s user 0.009s sys 0.008s
I20260811 12:47:25.549314 21294 tablet_bootstrap.cc:492] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b: No bootstrap required, opened a new log
I20260811 12:47:25.549489 21294 ts_tablet_manager.cc:1397] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.005s
I20260811 12:47:25.549628 21295 tablet_bootstrap.cc:492] T bafc890ef5dd4cbca5e81d0389051ed0 P fca017e7ad634088988f027f77db0b8f: No bootstrap required, opened a new log
I20260811 12:47:25.549810 21295 ts_tablet_manager.cc:1397] T bafc890ef5dd4cbca5e81d0389051ed0 P fca017e7ad634088988f027f77db0b8f: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.000s
I20260811 12:47:25.549947 21296 raft_consensus.cc:348] T bafc890ef5dd4cbca5e81d0389051ed0 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:25.550176 21296 raft_consensus.cc:374] T bafc890ef5dd4cbca5e81d0389051ed0 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:25.550273 21296 raft_consensus.cc:729] T bafc890ef5dd4cbca5e81d0389051ed0 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1a6c3187ba42caac4b7a4a6a30933d, State: Initialized, Role: FOLLOWER
I20260811 12:47:25.550384 21294 raft_consensus.cc:348] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:25.550623 21294 raft_consensus.cc:374] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:25.550582 21296 consensus_queue.cc:260] T bafc890ef5dd4cbca5e81d0389051ed0 P 3a1a6c3187ba42caac4b7a4a6a30933d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:25.550740 21294 raft_consensus.cc:729] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33f78dce92df4e1eb43f72d8e07ab67b, State: Initialized, Role: FOLLOWER
I20260811 12:47:25.551043 21295 raft_consensus.cc:348] T bafc890ef5dd4cbca5e81d0389051ed0 P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:25.551041 21294 consensus_queue.cc:260] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:25.551334 21295 raft_consensus.cc:374] T bafc890ef5dd4cbca5e81d0389051ed0 P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:25.551482 21295 raft_consensus.cc:729] T bafc890ef5dd4cbca5e81d0389051ed0 P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fca017e7ad634088988f027f77db0b8f, State: Initialized, Role: FOLLOWER
I20260811 12:47:25.551801 21295 consensus_queue.cc:260] T bafc890ef5dd4cbca5e81d0389051ed0 P fca017e7ad634088988f027f77db0b8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:25.552124 21294 ts_tablet_manager.cc:1428] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 12:47:25.552491 21295 ts_tablet_manager.cc:1428] T bafc890ef5dd4cbca5e81d0389051ed0 P fca017e7ad634088988f027f77db0b8f: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260811 12:47:25.554342 21296 ts_tablet_manager.cc:1428] T bafc890ef5dd4cbca5e81d0389051ed0 P 3a1a6c3187ba42caac4b7a4a6a30933d: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260811 12:47:25.597342 21336 raft_consensus.cc:482] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:47:25.597608 21336 raft_consensus.cc:504] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:25.598556 21336 leader_election.cc:288] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a1a6c3187ba42caac4b7a4a6a30933d (127.7.111.65:41091), fca017e7ad634088988f027f77db0b8f (127.7.111.67:43503)
I20260811 12:47:25.599131 21186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bafc890ef5dd4cbca5e81d0389051ed0" candidate_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fca017e7ad634088988f027f77db0b8f" is_pre_election: true
I20260811 12:47:25.599134 20653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bafc890ef5dd4cbca5e81d0389051ed0" candidate_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" is_pre_election: true
I20260811 12:47:25.599375 21186 raft_consensus.cc:2454] T bafc890ef5dd4cbca5e81d0389051ed0 P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33f78dce92df4e1eb43f72d8e07ab67b in term 0.
I20260811 12:47:25.599428 20653 raft_consensus.cc:2454] T bafc890ef5dd4cbca5e81d0389051ed0 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33f78dce92df4e1eb43f72d8e07ab67b in term 0.
I20260811 12:47:25.599846 20779 leader_election.cc:302] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33f78dce92df4e1eb43f72d8e07ab67b, fca017e7ad634088988f027f77db0b8f; no voters:
I20260811 12:47:25.600160 21336 raft_consensus.cc:2781] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:47:25.600304 21336 raft_consensus.cc:482] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:47:25.600400 21336 raft_consensus.cc:3037] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:25.603749 21336 raft_consensus.cc:504] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:25.604457 21336 leader_election.cc:288] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b [CANDIDATE]: Term 1 election: Requested vote from peers 3a1a6c3187ba42caac4b7a4a6a30933d (127.7.111.65:41091), fca017e7ad634088988f027f77db0b8f (127.7.111.67:43503)
I20260811 12:47:25.604959 21186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bafc890ef5dd4cbca5e81d0389051ed0" candidate_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fca017e7ad634088988f027f77db0b8f"
I20260811 12:47:25.604990 20653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bafc890ef5dd4cbca5e81d0389051ed0" candidate_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d"
I20260811 12:47:25.605162 21186 raft_consensus.cc:3037] T bafc890ef5dd4cbca5e81d0389051ed0 P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:25.605221 20653 raft_consensus.cc:3037] T bafc890ef5dd4cbca5e81d0389051ed0 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:25.607250 21186 raft_consensus.cc:2454] T bafc890ef5dd4cbca5e81d0389051ed0 P fca017e7ad634088988f027f77db0b8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33f78dce92df4e1eb43f72d8e07ab67b in term 1.
I20260811 12:47:25.607654 20779 leader_election.cc:302] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33f78dce92df4e1eb43f72d8e07ab67b, fca017e7ad634088988f027f77db0b8f; no voters:
I20260811 12:47:25.607985 21336 raft_consensus.cc:2781] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:47:25.608232 20653 raft_consensus.cc:2454] T bafc890ef5dd4cbca5e81d0389051ed0 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33f78dce92df4e1eb43f72d8e07ab67b in term 1.
I20260811 12:47:25.608150 21336 raft_consensus.cc:686] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b [term 1 LEADER]: Becoming Leader. State: Replica: 33f78dce92df4e1eb43f72d8e07ab67b, State: Running, Role: LEADER
I20260811 12:47:25.608737 21336 consensus_queue.cc:237] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:25.611582 20382 catalog_manager.cc:5495] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b reported cstate change: term changed from 0 to 1, leader changed from <none> to 33f78dce92df4e1eb43f72d8e07ab67b (127.7.111.66). New cstate: current_term: 1 leader_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } health_report { overall_health: UNKNOWN } } }
Aug 11 12:47:25 dist-test-slave-f0tn krb5kdc[20317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452442, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.126@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM
I20260811 12:47:25.665266 21368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:25.645238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:36621 (local address 127.0.0.1:52778)
0811 12:47:25.646049 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:25.646072 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:25.646226 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:25.651161 (+ 4935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:25.651166 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:25.652160 (+ 994us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:25.652611 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:25.652627 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.653977 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.653982 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:25.655030 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:25.655042 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.655872 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.655884 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:25.656075 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:25.656815 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:25.656838 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:25.659517 (+ 2679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:25.662648 (+ 3131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:25.662661 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:25.662669 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:25.663070 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:25.663419 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:25.663424 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:25.663428 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:25.663571 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:25.664094 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:25.664106 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:25.664306 (+ 200us) client_negotiation.cc:769] Sending connection context
0811 12:47:25.664511 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 12:47:25.664783 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":444,"threads_started":1}
I20260811 12:47:25.665307 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:25.647256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52778 (local address 127.7.111.126:36621)
0811 12:47:25.650795 (+ 3539us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:25.650801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:25.650827 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:47:25.650900 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:25.650906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:25.650985 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:25.651957 (+ 972us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:25.652807 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.653837 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.655160 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.655714 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.655834 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:25.659703 (+ 3869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:25.659708 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:25.659712 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:25.659741 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:25.662537 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:25.663175 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:25.663180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:25.663184 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:25.663309 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:25.663665 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:25.663671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:25.663674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:25.663957 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:25.664093 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:25.664601 (+ 508us) server_negotiation.cc:299] Negotiation successful
0811 12:47:25.664918 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":129,"server-negotiator.queue_time_us":3393,"thread_start_us":3163,"threads_started":1}
I20260811 12:47:25.670660 20382 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:52778:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 12:47:25.671818 20382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:47:25.678872 20617 tablet_service.cc:1467] Processing CreateTablet for tablet 875710a1fa384d63a55284fcf20bdf1a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8fe6e9446154452f98b16924e0753b33]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:47:25.679378 20858 tablet_service.cc:1467] Processing CreateTablet for tablet 875710a1fa384d63a55284fcf20bdf1a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8fe6e9446154452f98b16924e0753b33]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:47:25.679610 20617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 875710a1fa384d63a55284fcf20bdf1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:25.679935 20858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 875710a1fa384d63a55284fcf20bdf1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:25.686748 21139 tablet_service.cc:1467] Processing CreateTablet for tablet 875710a1fa384d63a55284fcf20bdf1a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8fe6e9446154452f98b16924e0753b33]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:47:25.687281 21139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 875710a1fa384d63a55284fcf20bdf1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:25.691231 21296 tablet_bootstrap.cc:492] T 875710a1fa384d63a55284fcf20bdf1a P 3a1a6c3187ba42caac4b7a4a6a30933d: Bootstrap starting.
I20260811 12:47:25.692679 21295 tablet_bootstrap.cc:492] T 875710a1fa384d63a55284fcf20bdf1a P fca017e7ad634088988f027f77db0b8f: Bootstrap starting.
I20260811 12:47:25.693018 21296 tablet_bootstrap.cc:654] T 875710a1fa384d63a55284fcf20bdf1a P 3a1a6c3187ba42caac4b7a4a6a30933d: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:25.694366 21295 tablet_bootstrap.cc:654] T 875710a1fa384d63a55284fcf20bdf1a P fca017e7ad634088988f027f77db0b8f: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:25.699436 21294 tablet_bootstrap.cc:492] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b: Bootstrap starting.
I20260811 12:47:25.701198 21294 tablet_bootstrap.cc:654] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:25.707548 21295 tablet_bootstrap.cc:492] T 875710a1fa384d63a55284fcf20bdf1a P fca017e7ad634088988f027f77db0b8f: No bootstrap required, opened a new log
I20260811 12:47:25.707719 21295 ts_tablet_manager.cc:1397] T 875710a1fa384d63a55284fcf20bdf1a P fca017e7ad634088988f027f77db0b8f: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.009s
I20260811 12:47:25.708520 21295 raft_consensus.cc:348] T 875710a1fa384d63a55284fcf20bdf1a P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:25.708774 21295 raft_consensus.cc:374] T 875710a1fa384d63a55284fcf20bdf1a P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:25.708855 21295 raft_consensus.cc:729] T 875710a1fa384d63a55284fcf20bdf1a P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fca017e7ad634088988f027f77db0b8f, State: Initialized, Role: FOLLOWER
I20260811 12:47:25.709139 21295 consensus_queue.cc:260] T 875710a1fa384d63a55284fcf20bdf1a P fca017e7ad634088988f027f77db0b8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:25.709646 21305 tablet_replica.cc:439] T 875710a1fa384d63a55284fcf20bdf1a P fca017e7ad634088988f027f77db0b8f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } } }
I20260811 12:47:25.710052 21305 tablet_replica.cc:442] T 875710a1fa384d63a55284fcf20bdf1a P fca017e7ad634088988f027f77db0b8f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:47:25.713871 21295 ts_tablet_manager.cc:1428] T 875710a1fa384d63a55284fcf20bdf1a P fca017e7ad634088988f027f77db0b8f: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260811 12:47:25.714556 21296 tablet_bootstrap.cc:492] T 875710a1fa384d63a55284fcf20bdf1a P 3a1a6c3187ba42caac4b7a4a6a30933d: No bootstrap required, opened a new log
I20260811 12:47:25.714751 21296 ts_tablet_manager.cc:1397] T 875710a1fa384d63a55284fcf20bdf1a P 3a1a6c3187ba42caac4b7a4a6a30933d: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.004s
I20260811 12:47:25.714929 21294 tablet_bootstrap.cc:492] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b: No bootstrap required, opened a new log
I20260811 12:47:25.715101 21294 ts_tablet_manager.cc:1397] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.006s
I20260811 12:47:25.715512 21296 raft_consensus.cc:348] T 875710a1fa384d63a55284fcf20bdf1a P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:25.715772 21296 raft_consensus.cc:374] T 875710a1fa384d63a55284fcf20bdf1a P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:25.715857 21296 raft_consensus.cc:729] T 875710a1fa384d63a55284fcf20bdf1a P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1a6c3187ba42caac4b7a4a6a30933d, State: Initialized, Role: FOLLOWER
I20260811 12:47:25.715842 21294 raft_consensus.cc:348] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:25.716089 21294 raft_consensus.cc:374] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:25.716171 21294 raft_consensus.cc:729] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33f78dce92df4e1eb43f72d8e07ab67b, State: Initialized, Role: FOLLOWER
I20260811 12:47:25.716091 21296 consensus_queue.cc:260] T 875710a1fa384d63a55284fcf20bdf1a P 3a1a6c3187ba42caac4b7a4a6a30933d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:25.716451 21294 consensus_queue.cc:260] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:25.716634 21296 ts_tablet_manager.cc:1428] T 875710a1fa384d63a55284fcf20bdf1a P 3a1a6c3187ba42caac4b7a4a6a30933d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 12:47:25.716540 21306 tablet_replica.cc:439] T 875710a1fa384d63a55284fcf20bdf1a P 3a1a6c3187ba42caac4b7a4a6a30933d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } } }
I20260811 12:47:25.716909 21306 tablet_replica.cc:442] T 875710a1fa384d63a55284fcf20bdf1a P 3a1a6c3187ba42caac4b7a4a6a30933d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:47:25.716912 21336 tablet_replica.cc:439] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } } }
I20260811 12:47:25.717178 21336 tablet_replica.cc:442] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:47:25.717917 21294 ts_tablet_manager.cc:1428] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 12:47:25.804065 21336 raft_consensus.cc:482] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:47:25.804327 21336 raft_consensus.cc:504] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:25.805233 21336 leader_election.cc:288] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a1a6c3187ba42caac4b7a4a6a30933d (127.7.111.65:41091), fca017e7ad634088988f027f77db0b8f (127.7.111.67:43503)
I20260811 12:47:25.805822 20653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "875710a1fa384d63a55284fcf20bdf1a" candidate_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" is_pre_election: true
I20260811 12:47:25.805823 21186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "875710a1fa384d63a55284fcf20bdf1a" candidate_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fca017e7ad634088988f027f77db0b8f" is_pre_election: true
I20260811 12:47:25.806113 21186 raft_consensus.cc:2454] T 875710a1fa384d63a55284fcf20bdf1a P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33f78dce92df4e1eb43f72d8e07ab67b in term 0.
I20260811 12:47:25.806113 20653 raft_consensus.cc:2454] T 875710a1fa384d63a55284fcf20bdf1a P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33f78dce92df4e1eb43f72d8e07ab67b in term 0.
I20260811 12:47:25.806670 20779 leader_election.cc:302] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33f78dce92df4e1eb43f72d8e07ab67b, fca017e7ad634088988f027f77db0b8f; no voters:
I20260811 12:47:25.807022 21336 raft_consensus.cc:2781] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:47:25.807132 21336 raft_consensus.cc:482] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:47:25.807261 21336 raft_consensus.cc:3037] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:25.809815 21336 raft_consensus.cc:504] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:25.810539 21336 leader_election.cc:288] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b [CANDIDATE]: Term 1 election: Requested vote from peers 3a1a6c3187ba42caac4b7a4a6a30933d (127.7.111.65:41091), fca017e7ad634088988f027f77db0b8f (127.7.111.67:43503)
I20260811 12:47:25.811043 20653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "875710a1fa384d63a55284fcf20bdf1a" candidate_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d"
I20260811 12:47:25.811308 20653 raft_consensus.cc:3037] T 875710a1fa384d63a55284fcf20bdf1a P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:25.811333 21186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "875710a1fa384d63a55284fcf20bdf1a" candidate_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fca017e7ad634088988f027f77db0b8f"
I20260811 12:47:25.811563 21186 raft_consensus.cc:3037] T 875710a1fa384d63a55284fcf20bdf1a P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:25.814276 20653 raft_consensus.cc:2454] T 875710a1fa384d63a55284fcf20bdf1a P 3a1a6c3187ba42caac4b7a4a6a30933d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33f78dce92df4e1eb43f72d8e07ab67b in term 1.
I20260811 12:47:25.814540 21186 raft_consensus.cc:2454] T 875710a1fa384d63a55284fcf20bdf1a P fca017e7ad634088988f027f77db0b8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33f78dce92df4e1eb43f72d8e07ab67b in term 1.
I20260811 12:47:25.814821 20782 leader_election.cc:302] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33f78dce92df4e1eb43f72d8e07ab67b, 3a1a6c3187ba42caac4b7a4a6a30933d; no voters:
I20260811 12:47:25.815294 21336 raft_consensus.cc:2781] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:47:25.815608 21336 raft_consensus.cc:686] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b [term 1 LEADER]: Becoming Leader. State: Replica: 33f78dce92df4e1eb43f72d8e07ab67b, State: Running, Role: LEADER
I20260811 12:47:25.815879 21336 consensus_queue.cc:237] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:25.817469 21322 tablet_replica.cc:439] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b: TxnStatusTablet state changed. Reason: New leader 33f78dce92df4e1eb43f72d8e07ab67b. Latest consensus state: current_term: 1 leader_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } } }
I20260811 12:47:25.817930 21322 tablet_replica.cc:442] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:47:25.820145 20382 catalog_manager.cc:5495] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b reported cstate change: term changed from 0 to 1, leader changed from <none> to 33f78dce92df4e1eb43f72d8e07ab67b (127.7.111.66). New cstate: current_term: 1 leader_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } health_report { overall_health: UNKNOWN } } }
I20260811 12:47:25.830281 21379 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 12:47:25.830564 21379 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 12:47:25.886731 21368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:25.879344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:46079 (local address 127.0.0.1:33672)
0811 12:47:25.879634 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:25.879662 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:25.879856 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:25.880663 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:25.880668 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:25.880693 (+ 25us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:47:25.880881 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:25.880892 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.882524 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.882530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:25.884159 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:25.884176 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.885269 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.885275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:25.885515 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:25.885598 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:47:25.886119 (+ 521us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:47:25.886142 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 12:47:25.886210 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 12:47:25.886370 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 12:47:25.887434 21288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:25.879859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33672 (local address 127.7.111.66:46079)
0811 12:47:25.880088 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:25.880095 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:25.880117 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:47:25.880190 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:25.880195 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:25.880278 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:25.880371 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:47:25.881063 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.882316 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.884341 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.885014 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.885136 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:25.885823 (+ 687us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:47:25.885992 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:47:25.886069 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:25.887051 (+ 982us) server_negotiation.cc:299] Negotiation successful
0811 12:47:25.887136 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 12:47:25.890983 20653 raft_consensus.cc:1261] T 875710a1fa384d63a55284fcf20bdf1a P 3a1a6c3187ba42caac4b7a4a6a30933d [term 1 FOLLOWER]: Refusing update from remote peer 33f78dce92df4e1eb43f72d8e07ab67b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:47:25.891124 21186 raft_consensus.cc:1261] T 875710a1fa384d63a55284fcf20bdf1a P fca017e7ad634088988f027f77db0b8f [term 1 FOLLOWER]: Refusing update from remote peer 33f78dce92df4e1eb43f72d8e07ab67b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:47:25.891719 21336 consensus_queue.cc:1035] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:47:25.892026 21322 consensus_queue.cc:1035] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:47:25.905344 21306 tablet_replica.cc:439] T 875710a1fa384d63a55284fcf20bdf1a P 3a1a6c3187ba42caac4b7a4a6a30933d: TxnStatusTablet state changed. Reason: New leader 33f78dce92df4e1eb43f72d8e07ab67b. Latest consensus state: current_term: 1 leader_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } } }
I20260811 12:47:25.905740 21306 tablet_replica.cc:442] T 875710a1fa384d63a55284fcf20bdf1a P 3a1a6c3187ba42caac4b7a4a6a30933d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:47:25.926141 21322 tablet_replica.cc:439] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } } }
I20260811 12:47:25.926481 21322 tablet_replica.cc:442] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:47:25.926625 21305 tablet_replica.cc:439] T 875710a1fa384d63a55284fcf20bdf1a P fca017e7ad634088988f027f77db0b8f: TxnStatusTablet state changed. Reason: New leader 33f78dce92df4e1eb43f72d8e07ab67b. Latest consensus state: current_term: 1 leader_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } } }
I20260811 12:47:25.926909 21305 tablet_replica.cc:442] T 875710a1fa384d63a55284fcf20bdf1a P fca017e7ad634088988f027f77db0b8f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:47:25.931233 21306 tablet_replica.cc:439] T 875710a1fa384d63a55284fcf20bdf1a P 3a1a6c3187ba42caac4b7a4a6a30933d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } } }
I20260811 12:47:25.931509 21306 tablet_replica.cc:442] T 875710a1fa384d63a55284fcf20bdf1a P 3a1a6c3187ba42caac4b7a4a6a30933d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:47:25.936614 21303 tablet_replica.cc:439] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } } }
I20260811 12:47:25.936925 21303 tablet_replica.cc:442] T 875710a1fa384d63a55284fcf20bdf1a P 33f78dce92df4e1eb43f72d8e07ab67b: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:47:25.938927 21305 tablet_replica.cc:439] T 875710a1fa384d63a55284fcf20bdf1a P fca017e7ad634088988f027f77db0b8f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } } }
I20260811 12:47:25.939260 21305 tablet_replica.cc:442] T 875710a1fa384d63a55284fcf20bdf1a P fca017e7ad634088988f027f77db0b8f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:47:25.969580 21288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:25.963515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:39391 (local address 127.7.111.66:46079)
0811 12:47:25.963712 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:25.963718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:25.964401 (+ 683us) server_negotiation.cc:407] Connection header received
0811 12:47:25.964716 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:25.964721 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:25.964786 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:25.964858 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:47:25.965373 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.966190 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.967689 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.968146 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.968256 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:25.968783 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:47:25.968889 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:47:25.968961 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:25.969135 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 12:47:25.969240 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":73}
I20260811 12:47:25.969820 21389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:25.963515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:46079 (local address 127.7.111.66:39391)
0811 12:47:25.964285 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:25.964313 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:25.964522 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:25.964890 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:25.964894 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:25.964905 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:47:25.965224 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:25.965234 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.966320 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.966324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:25.967556 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:25.967569 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:25.968355 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:25.968363 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:25.968517 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:25.968597 (+ 80us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:47:25.968981 (+ 384us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:47:25.968992 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 12:47:25.969065 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 12:47:25.969240 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"mutex_wait_us":26,"thread_start_us":404,"threads_started":1}
I20260811 12:47:25.980492 21185 raft_consensus.cc:1261] T bafc890ef5dd4cbca5e81d0389051ed0 P fca017e7ad634088988f027f77db0b8f [term 1 FOLLOWER]: Refusing update from remote peer 33f78dce92df4e1eb43f72d8e07ab67b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:47:25.981102 20653 raft_consensus.cc:1261] T bafc890ef5dd4cbca5e81d0389051ed0 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 1 FOLLOWER]: Refusing update from remote peer 33f78dce92df4e1eb43f72d8e07ab67b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:47:25.980998 21322 consensus_queue.cc:1035] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:47:25.981765 21322 consensus_queue.cc:1035] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:47:26.065939 20383 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:47:26.066195 20383 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:47:26.069347 20383 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c: Sending DeleteTablet for 3 replicas of tablet bafc890ef5dd4cbca5e81d0389051ed0
I20260811 12:47:26.070081 20617 tablet_service.cc:1514] Processing DeleteTablet for tablet bafc890ef5dd4cbca5e81d0389051ed0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:47:26 UTC) from {username='oryx', principal='oryx/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:36538
I20260811 12:47:26.070633 21394 tablet_replica.cc:330] T bafc890ef5dd4cbca5e81d0389051ed0 P 3a1a6c3187ba42caac4b7a4a6a30933d: stopping tablet replica
I20260811 12:47:26.070915 21394 raft_consensus.cc:2229] T bafc890ef5dd4cbca5e81d0389051ed0 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:47:26.071103 21139 tablet_service.cc:1514] Processing DeleteTablet for tablet bafc890ef5dd4cbca5e81d0389051ed0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:47:26 UTC) from {username='oryx', principal='oryx/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:33280
I20260811 12:47:26.071277 21394 raft_consensus.cc:2258] T bafc890ef5dd4cbca5e81d0389051ed0 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:47:26.072371 21394 ts_tablet_manager.cc:1905] T bafc890ef5dd4cbca5e81d0389051ed0 P 3a1a6c3187ba42caac4b7a4a6a30933d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:47:26.077811 20862 tablet_service.cc:1514] Processing DeleteTablet for tablet bafc890ef5dd4cbca5e81d0389051ed0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:47:26 UTC) from {username='oryx', principal='oryx/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:33648
I20260811 12:47:26.078967 21394 ts_tablet_manager.cc:1918] T bafc890ef5dd4cbca5e81d0389051ed0 P 3a1a6c3187ba42caac4b7a4a6a30933d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:47:26.079159 21394 log.cc:1196] T bafc890ef5dd4cbca5e81d0389051ed0 P 3a1a6c3187ba42caac4b7a4a6a30933d: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/bafc890ef5dd4cbca5e81d0389051ed0
I20260811 12:47:26.079697 21394 ts_tablet_manager.cc:1939] T bafc890ef5dd4cbca5e81d0389051ed0 P 3a1a6c3187ba42caac4b7a4a6a30933d: Deleting consensus metadata
I20260811 12:47:26.080809 20367 catalog_manager.cc:4841] TS 3a1a6c3187ba42caac4b7a4a6a30933d (127.7.111.65:41091): tablet bafc890ef5dd4cbca5e81d0389051ed0 (table test-table [id=f2a8f8a4e3f44410b2593aeae1da31f6]) successfully deleted
I20260811 12:47:26.089849 21395 tablet_replica.cc:330] T bafc890ef5dd4cbca5e81d0389051ed0 P fca017e7ad634088988f027f77db0b8f: stopping tablet replica
I20260811 12:47:26.090468 21396 tablet_replica.cc:330] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b: stopping tablet replica
I20260811 12:47:26.090723 21396 raft_consensus.cc:2229] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:47:26.091187 21396 raft_consensus.cc:2258] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:47:26.091601 21395 raft_consensus.cc:2229] T bafc890ef5dd4cbca5e81d0389051ed0 P fca017e7ad634088988f027f77db0b8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:47:26.091951 21395 raft_consensus.cc:2258] T bafc890ef5dd4cbca5e81d0389051ed0 P fca017e7ad634088988f027f77db0b8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:47:26.092988 21396 ts_tablet_manager.cc:1905] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:47:26.095201 21395 ts_tablet_manager.cc:1905] T bafc890ef5dd4cbca5e81d0389051ed0 P fca017e7ad634088988f027f77db0b8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:47:26.101044 21396 ts_tablet_manager.cc:1918] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:47:26.101295 21396 log.cc:1196] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/bafc890ef5dd4cbca5e81d0389051ed0
I20260811 12:47:26.101797 21395 ts_tablet_manager.cc:1918] T bafc890ef5dd4cbca5e81d0389051ed0 P fca017e7ad634088988f027f77db0b8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:47:26.101965 21395 log.cc:1196] T bafc890ef5dd4cbca5e81d0389051ed0 P fca017e7ad634088988f027f77db0b8f: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/bafc890ef5dd4cbca5e81d0389051ed0
I20260811 12:47:26.101985 21396 ts_tablet_manager.cc:1939] T bafc890ef5dd4cbca5e81d0389051ed0 P 33f78dce92df4e1eb43f72d8e07ab67b: Deleting consensus metadata
I20260811 12:47:26.102370 21395 ts_tablet_manager.cc:1939] T bafc890ef5dd4cbca5e81d0389051ed0 P fca017e7ad634088988f027f77db0b8f: Deleting consensus metadata
I20260811 12:47:26.103478 20364 catalog_manager.cc:4841] TS fca017e7ad634088988f027f77db0b8f (127.7.111.67:43503): tablet bafc890ef5dd4cbca5e81d0389051ed0 (table test-table [id=f2a8f8a4e3f44410b2593aeae1da31f6]) successfully deleted
I20260811 12:47:26.103488 20366 catalog_manager.cc:4841] TS 33f78dce92df4e1eb43f72d8e07ab67b (127.7.111.66:46079): tablet bafc890ef5dd4cbca5e81d0389051ed0 (table test-table [id=f2a8f8a4e3f44410b2593aeae1da31f6]) successfully deleted
Aug 11 12:47:26 dist-test-slave-f0tn krb5kdc[20317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 12:47:26 dist-test-slave-f0tn krb5kdc[20317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:26 dist-test-slave-f0tn krb5kdc[20317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 12:47:26.153427 21370 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:26.140276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52790 (local address 127.7.111.126:36621)
0811 12:47:26.140504 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:26.140510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:26.140896 (+ 386us) server_negotiation.cc:407] Connection header received
0811 12:47:26.141166 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:26.141178 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:26.141252 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:26.141345 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:26.143339 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.144146 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.146099 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.146648 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.146754 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:26.153127 (+ 6373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52790: BlockingRecv error: recv got EOF from 127.0.0.1:52790 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 11 12:47:26 dist-test-slave-f0tn krb5kdc[20317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM
I20260811 12:47:26.171231 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:26.156496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52800 (local address 127.7.111.126:36621)
0811 12:47:26.156684 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:26.156691 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:26.157009 (+ 318us) server_negotiation.cc:407] Connection header received
0811 12:47:26.157182 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:26.157188 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:26.157278 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:26.157434 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:26.158656 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.159799 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.161031 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.161734 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.161896 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:26.165799 (+ 3903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:26.165804 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:26.165808 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:26.165843 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:26.168425 (+ 2582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:26.169301 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:26.169305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:26.169308 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:26.169501 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:26.169907 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:26.169910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:26.169912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:26.170169 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:26.170379 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:26.170728 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 12:47:26.170933 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:47:26.174765 20383 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:47:26.175369 20383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:47:26.182205 20617 tablet_service.cc:1467] Processing CreateTablet for tablet 765ff54ab42849f6a70e16bca84af9a0 (DEFAULT_TABLE table=test-table [id=2154426edf2d4a9e88154ac5a03a2fe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:47:26.182644 20617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 765ff54ab42849f6a70e16bca84af9a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:26.182754 20862 tablet_service.cc:1467] Processing CreateTablet for tablet 765ff54ab42849f6a70e16bca84af9a0 (DEFAULT_TABLE table=test-table [id=2154426edf2d4a9e88154ac5a03a2fe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:47:26.182804 21139 tablet_service.cc:1467] Processing CreateTablet for tablet 765ff54ab42849f6a70e16bca84af9a0 (DEFAULT_TABLE table=test-table [id=2154426edf2d4a9e88154ac5a03a2fe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:47:26.183182 20862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 765ff54ab42849f6a70e16bca84af9a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:26.183354 21139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 765ff54ab42849f6a70e16bca84af9a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:26.189646 21296 tablet_bootstrap.cc:492] T 765ff54ab42849f6a70e16bca84af9a0 P 3a1a6c3187ba42caac4b7a4a6a30933d: Bootstrap starting.
I20260811 12:47:26.189913 21294 tablet_bootstrap.cc:492] T 765ff54ab42849f6a70e16bca84af9a0 P 33f78dce92df4e1eb43f72d8e07ab67b: Bootstrap starting.
I20260811 12:47:26.191125 21296 tablet_bootstrap.cc:654] T 765ff54ab42849f6a70e16bca84af9a0 P 3a1a6c3187ba42caac4b7a4a6a30933d: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:26.191870 21294 tablet_bootstrap.cc:654] T 765ff54ab42849f6a70e16bca84af9a0 P 33f78dce92df4e1eb43f72d8e07ab67b: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:26.193436 21296 tablet_bootstrap.cc:492] T 765ff54ab42849f6a70e16bca84af9a0 P 3a1a6c3187ba42caac4b7a4a6a30933d: No bootstrap required, opened a new log
I20260811 12:47:26.193589 21296 ts_tablet_manager.cc:1397] T 765ff54ab42849f6a70e16bca84af9a0 P 3a1a6c3187ba42caac4b7a4a6a30933d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 12:47:26.194360 21296 raft_consensus.cc:348] T 765ff54ab42849f6a70e16bca84af9a0 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:26.194710 21296 raft_consensus.cc:374] T 765ff54ab42849f6a70e16bca84af9a0 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:26.194772 21295 tablet_bootstrap.cc:492] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f: Bootstrap starting.
I20260811 12:47:26.194871 21296 raft_consensus.cc:729] T 765ff54ab42849f6a70e16bca84af9a0 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1a6c3187ba42caac4b7a4a6a30933d, State: Initialized, Role: FOLLOWER
I20260811 12:47:26.195142 21296 consensus_queue.cc:260] T 765ff54ab42849f6a70e16bca84af9a0 P 3a1a6c3187ba42caac4b7a4a6a30933d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:26.196275 21294 tablet_bootstrap.cc:492] T 765ff54ab42849f6a70e16bca84af9a0 P 33f78dce92df4e1eb43f72d8e07ab67b: No bootstrap required, opened a new log
I20260811 12:47:26.196439 21295 tablet_bootstrap.cc:654] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:26.196442 21294 ts_tablet_manager.cc:1397] T 765ff54ab42849f6a70e16bca84af9a0 P 33f78dce92df4e1eb43f72d8e07ab67b: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260811 12:47:26.197337 21294 raft_consensus.cc:348] T 765ff54ab42849f6a70e16bca84af9a0 P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:26.197669 21294 raft_consensus.cc:374] T 765ff54ab42849f6a70e16bca84af9a0 P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:26.198829 21294 raft_consensus.cc:729] T 765ff54ab42849f6a70e16bca84af9a0 P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33f78dce92df4e1eb43f72d8e07ab67b, State: Initialized, Role: FOLLOWER
I20260811 12:47:26.198873 21296 ts_tablet_manager.cc:1428] T 765ff54ab42849f6a70e16bca84af9a0 P 3a1a6c3187ba42caac4b7a4a6a30933d: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260811 12:47:26.202454 21295 tablet_bootstrap.cc:492] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f: No bootstrap required, opened a new log
I20260811 12:47:26.202601 21295 ts_tablet_manager.cc:1397] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260811 12:47:26.203209 21295 raft_consensus.cc:348] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:26.203425 21295 raft_consensus.cc:374] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:26.203544 21295 raft_consensus.cc:729] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fca017e7ad634088988f027f77db0b8f, State: Initialized, Role: FOLLOWER
I20260811 12:47:26.203764 21295 consensus_queue.cc:260] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:26.204272 21295 ts_tablet_manager.cc:1428] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 12:47:26.205811 21294 consensus_queue.cc:260] T 765ff54ab42849f6a70e16bca84af9a0 P 33f78dce92df4e1eb43f72d8e07ab67b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:26.206529 21294 ts_tablet_manager.cc:1428] T 765ff54ab42849f6a70e16bca84af9a0 P 33f78dce92df4e1eb43f72d8e07ab67b: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260811 12:47:26.231129 21305 raft_consensus.cc:482] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:47:26.231372 21305 raft_consensus.cc:504] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:26.233254 21305 leader_election.cc:288] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a1a6c3187ba42caac4b7a4a6a30933d (127.7.111.65:41091), 33f78dce92df4e1eb43f72d8e07ab67b (127.7.111.66:46079)
I20260811 12:47:26.254705 21288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:26.234341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:55981 (local address 127.7.111.66:46079)
0811 12:47:26.234551 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:26.234558 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:26.249004 (+ 14446us) server_negotiation.cc:407] Connection header received
0811 12:47:26.249191 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:26.249195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:26.249257 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:26.249330 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:47:26.249910 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.250644 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.252481 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.253790 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.253947 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:26.254028 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:26.254311 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 12:47:26.254447 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:47:26.254951 21423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:26.233802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:46079 (local address 127.7.111.67:55981)
0811 12:47:26.248893 (+ 15091us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:26.248908 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:26.249039 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:26.249397 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:26.249402 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:26.249432 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:47:26.249740 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:26.249749 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.250781 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.250786 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:26.252256 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:26.252272 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.253960 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.253966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:26.254161 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:26.254186 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 12:47:26.254267 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 12:47:26.254389 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14986,"mutex_wait_us":2,"thread_start_us":13948,"threads_started":1}
I20260811 12:47:26.255843 20905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "765ff54ab42849f6a70e16bca84af9a0" candidate_uuid: "fca017e7ad634088988f027f77db0b8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" is_pre_election: true
I20260811 12:47:26.256276 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:26.233755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:35489 (local address 127.7.111.65:41091)
0811 12:47:26.247604 (+ 13849us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:26.247612 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:26.248189 (+ 577us) server_negotiation.cc:407] Connection header received
0811 12:47:26.248404 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:26.248410 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:26.248500 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:26.248623 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:47:26.249632 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.250817 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.253530 (+ 2713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.254941 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.255166 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:26.255249 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:26.255806 (+ 557us) server_negotiation.cc:299] Negotiation successful
0811 12:47:26.255949 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13690,"thread_start_us":14596,"threads_started":1}
I20260811 12:47:26.256316 21427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:26.233578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:41091 (local address 127.7.111.67:35489)
0811 12:47:26.248046 (+ 14468us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:26.248069 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:26.248244 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:26.248826 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:26.248832 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:26.248875 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:47:26.249326 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:26.249341 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.251044 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.251055 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:26.253297 (+ 2242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:26.253315 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.255410 (+ 2095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.255417 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:26.255600 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:26.255638 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 12:47:26.255732 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 12:47:26.255869 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14311,"mutex_wait_us":27,"spinlock_wait_cycles":3328,"thread_start_us":15013,"threads_started":1}
I20260811 12:47:26.256395 20905 raft_consensus.cc:2454] T 765ff54ab42849f6a70e16bca84af9a0 P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fca017e7ad634088988f027f77db0b8f in term 0.
I20260811 12:47:26.257061 20647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "765ff54ab42849f6a70e16bca84af9a0" candidate_uuid: "fca017e7ad634088988f027f77db0b8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" is_pre_election: true
I20260811 12:47:26.257133 21055 leader_election.cc:302] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33f78dce92df4e1eb43f72d8e07ab67b, fca017e7ad634088988f027f77db0b8f; no voters:
I20260811 12:47:26.257406 20647 raft_consensus.cc:2454] T 765ff54ab42849f6a70e16bca84af9a0 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fca017e7ad634088988f027f77db0b8f in term 0.
I20260811 12:47:26.257568 21305 raft_consensus.cc:2781] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:47:26.257697 21305 raft_consensus.cc:482] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:47:26.257788 21305 raft_consensus.cc:3037] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:26.260092 21305 raft_consensus.cc:504] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:26.260659 21305 leader_election.cc:288] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f [CANDIDATE]: Term 1 election: Requested vote from peers 3a1a6c3187ba42caac4b7a4a6a30933d (127.7.111.65:41091), 33f78dce92df4e1eb43f72d8e07ab67b (127.7.111.66:46079)
I20260811 12:47:26.261113 20647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "765ff54ab42849f6a70e16bca84af9a0" candidate_uuid: "fca017e7ad634088988f027f77db0b8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d"
I20260811 12:47:26.261317 20647 raft_consensus.cc:3037] T 765ff54ab42849f6a70e16bca84af9a0 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:26.261282 20905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "765ff54ab42849f6a70e16bca84af9a0" candidate_uuid: "fca017e7ad634088988f027f77db0b8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33f78dce92df4e1eb43f72d8e07ab67b"
I20260811 12:47:26.261547 20905 raft_consensus.cc:3037] T 765ff54ab42849f6a70e16bca84af9a0 P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:26.263441 20647 raft_consensus.cc:2454] T 765ff54ab42849f6a70e16bca84af9a0 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fca017e7ad634088988f027f77db0b8f in term 1.
I20260811 12:47:26.263823 21059 leader_election.cc:302] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a1a6c3187ba42caac4b7a4a6a30933d, fca017e7ad634088988f027f77db0b8f; no voters:
I20260811 12:47:26.264094 21305 raft_consensus.cc:2781] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:47:26.264568 21305 raft_consensus.cc:686] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f [term 1 LEADER]: Becoming Leader. State: Replica: fca017e7ad634088988f027f77db0b8f, State: Running, Role: LEADER
I20260811 12:47:26.265045 20905 raft_consensus.cc:2454] T 765ff54ab42849f6a70e16bca84af9a0 P 33f78dce92df4e1eb43f72d8e07ab67b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fca017e7ad634088988f027f77db0b8f in term 1.
I20260811 12:47:26.265034 21305 consensus_queue.cc:237] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:26.268786 20382 catalog_manager.cc:5495] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f reported cstate change: term changed from 0 to 1, leader changed from <none> to fca017e7ad634088988f027f77db0b8f (127.7.111.67). New cstate: current_term: 1 leader_uuid: "fca017e7ad634088988f027f77db0b8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } health_report { overall_health: HEALTHY } } }
I20260811 12:47:26.294365 21435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:26.287809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33292 (local address 127.7.111.67:43503)
0811 12:47:26.288369 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:26.288373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:26.288401 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:47:26.288484 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:26.288487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:26.288571 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:26.288721 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:47:26.289171 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.289982 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.291991 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.292427 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.292516 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:26.293062 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:47:26.293735 (+ 673us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:47:26.293842 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:26.293978 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 12:47:26.294042 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":300,"threads_started":1}
I20260811 12:47:26.298180 20647 raft_consensus.cc:1261] T 765ff54ab42849f6a70e16bca84af9a0 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 1 FOLLOWER]: Refusing update from remote peer fca017e7ad634088988f027f77db0b8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:47:26.298611 20905 raft_consensus.cc:1261] T 765ff54ab42849f6a70e16bca84af9a0 P 33f78dce92df4e1eb43f72d8e07ab67b [term 1 FOLLOWER]: Refusing update from remote peer fca017e7ad634088988f027f77db0b8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:47:26.298868 21428 consensus_queue.cc:1035] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:47:26.299513 21305 consensus_queue.cc:1035] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:47:26.304782 21329 mvcc.cc:205] Tried to move back new op lower bound from 7317309220030509056 to 7317309219907108864. Current Snapshot: MvccSnapshot[applied={T|T < 7317309220030509056}]
I20260811 12:47:26.320504 20383 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:47:26.320806 20383 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:47:26.324846 20383 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c: Sending DeleteTablet for 3 replicas of tablet 765ff54ab42849f6a70e16bca84af9a0
I20260811 12:47:26.325767 20862 tablet_service.cc:1514] Processing DeleteTablet for tablet 765ff54ab42849f6a70e16bca84af9a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:47:26 UTC) from {username='oryx', principal='oryx/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:33648
I20260811 12:47:26.326370 21396 tablet_replica.cc:330] T 765ff54ab42849f6a70e16bca84af9a0 P 33f78dce92df4e1eb43f72d8e07ab67b: stopping tablet replica
I20260811 12:47:26.326385 20617 tablet_service.cc:1514] Processing DeleteTablet for tablet 765ff54ab42849f6a70e16bca84af9a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:47:26 UTC) from {username='oryx', principal='oryx/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:36538
I20260811 12:47:26.326622 21396 raft_consensus.cc:2229] T 765ff54ab42849f6a70e16bca84af9a0 P 33f78dce92df4e1eb43f72d8e07ab67b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:47:26.326669 21394 tablet_replica.cc:330] T 765ff54ab42849f6a70e16bca84af9a0 P 3a1a6c3187ba42caac4b7a4a6a30933d: stopping tablet replica
I20260811 12:47:26.326864 21394 raft_consensus.cc:2229] T 765ff54ab42849f6a70e16bca84af9a0 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:47:26.326872 21396 raft_consensus.cc:2258] T 765ff54ab42849f6a70e16bca84af9a0 P 33f78dce92df4e1eb43f72d8e07ab67b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:47:26.327081 21394 raft_consensus.cc:2258] T 765ff54ab42849f6a70e16bca84af9a0 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:47:26.328186 21394 ts_tablet_manager.cc:1905] T 765ff54ab42849f6a70e16bca84af9a0 P 3a1a6c3187ba42caac4b7a4a6a30933d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:47:26.328438 21139 tablet_service.cc:1514] Processing DeleteTablet for tablet 765ff54ab42849f6a70e16bca84af9a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:47:26 UTC) from {username='oryx', principal='oryx/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:33280
I20260811 12:47:26.328797 21395 tablet_replica.cc:330] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f: stopping tablet replica
I20260811 12:47:26.329031 21395 raft_consensus.cc:2229] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:47:26.329635 21395 raft_consensus.cc:2258] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:47:26.331620 20383 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:47:26.332350 20383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:47:26.334610 21396 ts_tablet_manager.cc:1905] T 765ff54ab42849f6a70e16bca84af9a0 P 33f78dce92df4e1eb43f72d8e07ab67b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:47:26.341135 20617 tablet_service.cc:1467] Processing CreateTablet for tablet a7699a7befe7482d89221fa6e9075838 (DEFAULT_TABLE table=test-table [id=bb7ae0bc99ab4066a23104cbcf017634]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:47:26.341638 20617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7699a7befe7482d89221fa6e9075838. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:26.342935 21394 ts_tablet_manager.cc:1918] T 765ff54ab42849f6a70e16bca84af9a0 P 3a1a6c3187ba42caac4b7a4a6a30933d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:47:26.343093 21394 log.cc:1196] T 765ff54ab42849f6a70e16bca84af9a0 P 3a1a6c3187ba42caac4b7a4a6a30933d: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/765ff54ab42849f6a70e16bca84af9a0
I20260811 12:47:26.343572 21394 ts_tablet_manager.cc:1939] T 765ff54ab42849f6a70e16bca84af9a0 P 3a1a6c3187ba42caac4b7a4a6a30933d: Deleting consensus metadata
I20260811 12:47:26.344759 20367 catalog_manager.cc:4841] TS 3a1a6c3187ba42caac4b7a4a6a30933d (127.7.111.65:41091): tablet 765ff54ab42849f6a70e16bca84af9a0 (table test-table [id=2154426edf2d4a9e88154ac5a03a2fe2]) successfully deleted
I20260811 12:47:26.345109 21395 ts_tablet_manager.cc:1905] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:47:26.345763 21139 tablet_service.cc:1467] Processing CreateTablet for tablet a7699a7befe7482d89221fa6e9075838 (DEFAULT_TABLE table=test-table [id=bb7ae0bc99ab4066a23104cbcf017634]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:47:26.346174 21139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7699a7befe7482d89221fa6e9075838. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:26.349086 21296 tablet_bootstrap.cc:492] T a7699a7befe7482d89221fa6e9075838 P 3a1a6c3187ba42caac4b7a4a6a30933d: Bootstrap starting.
I20260811 12:47:26.349826 20862 tablet_service.cc:1467] Processing CreateTablet for tablet a7699a7befe7482d89221fa6e9075838 (DEFAULT_TABLE table=test-table [id=bb7ae0bc99ab4066a23104cbcf017634]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:47:26.350320 20862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7699a7befe7482d89221fa6e9075838. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:26.350694 21296 tablet_bootstrap.cc:654] T a7699a7befe7482d89221fa6e9075838 P 3a1a6c3187ba42caac4b7a4a6a30933d: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:26.351022 21395 ts_tablet_manager.cc:1918] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:47:26.351174 21395 log.cc:1196] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/765ff54ab42849f6a70e16bca84af9a0
I20260811 12:47:26.351614 21395 ts_tablet_manager.cc:1939] T 765ff54ab42849f6a70e16bca84af9a0 P fca017e7ad634088988f027f77db0b8f: Deleting consensus metadata
I20260811 12:47:26.352805 21295 tablet_bootstrap.cc:492] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f: Bootstrap starting.
I20260811 12:47:26.352793 20364 catalog_manager.cc:4841] TS fca017e7ad634088988f027f77db0b8f (127.7.111.67:43503): tablet 765ff54ab42849f6a70e16bca84af9a0 (table test-table [id=2154426edf2d4a9e88154ac5a03a2fe2]) successfully deleted
I20260811 12:47:26.354737 21295 tablet_bootstrap.cc:654] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:26.359717 21296 tablet_bootstrap.cc:492] T a7699a7befe7482d89221fa6e9075838 P 3a1a6c3187ba42caac4b7a4a6a30933d: No bootstrap required, opened a new log
I20260811 12:47:26.359980 21296 ts_tablet_manager.cc:1397] T a7699a7befe7482d89221fa6e9075838 P 3a1a6c3187ba42caac4b7a4a6a30933d: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260811 12:47:26.360913 21296 raft_consensus.cc:348] T a7699a7befe7482d89221fa6e9075838 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:26.362555 21296 raft_consensus.cc:374] T a7699a7befe7482d89221fa6e9075838 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:26.362708 21296 raft_consensus.cc:729] T a7699a7befe7482d89221fa6e9075838 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1a6c3187ba42caac4b7a4a6a30933d, State: Initialized, Role: FOLLOWER
I20260811 12:47:26.363071 21296 consensus_queue.cc:260] T a7699a7befe7482d89221fa6e9075838 P 3a1a6c3187ba42caac4b7a4a6a30933d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:26.363437 21396 ts_tablet_manager.cc:1918] T 765ff54ab42849f6a70e16bca84af9a0 P 33f78dce92df4e1eb43f72d8e07ab67b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:47:26.363633 21396 log.cc:1196] T 765ff54ab42849f6a70e16bca84af9a0 P 33f78dce92df4e1eb43f72d8e07ab67b: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/765ff54ab42849f6a70e16bca84af9a0
I20260811 12:47:26.364125 21295 tablet_bootstrap.cc:492] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f: No bootstrap required, opened a new log
I20260811 12:47:26.364156 21396 ts_tablet_manager.cc:1939] T 765ff54ab42849f6a70e16bca84af9a0 P 33f78dce92df4e1eb43f72d8e07ab67b: Deleting consensus metadata
I20260811 12:47:26.364336 21295 ts_tablet_manager.cc:1397] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.010s
I20260811 12:47:26.365293 21295 raft_consensus.cc:348] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:26.365736 20366 catalog_manager.cc:4841] TS 33f78dce92df4e1eb43f72d8e07ab67b (127.7.111.66:46079): tablet 765ff54ab42849f6a70e16bca84af9a0 (table test-table [id=2154426edf2d4a9e88154ac5a03a2fe2]) successfully deleted
I20260811 12:47:26.365834 21294 tablet_bootstrap.cc:492] T a7699a7befe7482d89221fa6e9075838 P 33f78dce92df4e1eb43f72d8e07ab67b: Bootstrap starting.
I20260811 12:47:26.367748 21294 tablet_bootstrap.cc:654] T a7699a7befe7482d89221fa6e9075838 P 33f78dce92df4e1eb43f72d8e07ab67b: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:26.368903 21296 ts_tablet_manager.cc:1428] T a7699a7befe7482d89221fa6e9075838 P 3a1a6c3187ba42caac4b7a4a6a30933d: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260811 12:47:26.370290 21295 raft_consensus.cc:374] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:26.370432 21295 raft_consensus.cc:729] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fca017e7ad634088988f027f77db0b8f, State: Initialized, Role: FOLLOWER
I20260811 12:47:26.371269 21295 consensus_queue.cc:260] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:26.373208 21295 ts_tablet_manager.cc:1428] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20260811 12:47:26.387322 21294 tablet_bootstrap.cc:492] T a7699a7befe7482d89221fa6e9075838 P 33f78dce92df4e1eb43f72d8e07ab67b: No bootstrap required, opened a new log
I20260811 12:47:26.387648 21294 ts_tablet_manager.cc:1397] T a7699a7befe7482d89221fa6e9075838 P 33f78dce92df4e1eb43f72d8e07ab67b: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.005s
I20260811 12:47:26.389106 21294 raft_consensus.cc:348] T a7699a7befe7482d89221fa6e9075838 P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:26.389598 21294 raft_consensus.cc:374] T a7699a7befe7482d89221fa6e9075838 P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:26.389761 21294 raft_consensus.cc:729] T a7699a7befe7482d89221fa6e9075838 P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33f78dce92df4e1eb43f72d8e07ab67b, State: Initialized, Role: FOLLOWER
I20260811 12:47:26.390141 21294 consensus_queue.cc:260] T a7699a7befe7482d89221fa6e9075838 P 33f78dce92df4e1eb43f72d8e07ab67b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:26.391040 21294 ts_tablet_manager.cc:1428] T a7699a7befe7482d89221fa6e9075838 P 33f78dce92df4e1eb43f72d8e07ab67b: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 12:47:26.396005 21428 raft_consensus.cc:482] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:47:26.396251 21428 raft_consensus.cc:504] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:26.397069 21428 leader_election.cc:288] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a1a6c3187ba42caac4b7a4a6a30933d (127.7.111.65:41091), 33f78dce92df4e1eb43f72d8e07ab67b (127.7.111.66:46079)
I20260811 12:47:26.397794 20647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7699a7befe7482d89221fa6e9075838" candidate_uuid: "fca017e7ad634088988f027f77db0b8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" is_pre_election: true
I20260811 12:47:26.398099 20647 raft_consensus.cc:2454] T a7699a7befe7482d89221fa6e9075838 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fca017e7ad634088988f027f77db0b8f in term 0.
I20260811 12:47:26.398012 20902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7699a7befe7482d89221fa6e9075838" candidate_uuid: "fca017e7ad634088988f027f77db0b8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" is_pre_election: true
I20260811 12:47:26.398340 20902 raft_consensus.cc:2454] T a7699a7befe7482d89221fa6e9075838 P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fca017e7ad634088988f027f77db0b8f in term 0.
I20260811 12:47:26.398692 21059 leader_election.cc:302] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a1a6c3187ba42caac4b7a4a6a30933d, fca017e7ad634088988f027f77db0b8f; no voters:
I20260811 12:47:26.399137 21428 raft_consensus.cc:2781] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:47:26.399259 21428 raft_consensus.cc:482] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:47:26.399376 21428 raft_consensus.cc:3037] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:26.402885 21428 raft_consensus.cc:504] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:26.404122 20647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7699a7befe7482d89221fa6e9075838" candidate_uuid: "fca017e7ad634088988f027f77db0b8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d"
I20260811 12:47:26.404366 20647 raft_consensus.cc:3037] T a7699a7befe7482d89221fa6e9075838 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:26.407539 20647 raft_consensus.cc:2454] T a7699a7befe7482d89221fa6e9075838 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fca017e7ad634088988f027f77db0b8f in term 1.
I20260811 12:47:26.408087 21059 leader_election.cc:302] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a1a6c3187ba42caac4b7a4a6a30933d, fca017e7ad634088988f027f77db0b8f; no voters:
I20260811 12:47:26.408455 21305 raft_consensus.cc:2781] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:47:26.411556 21428 leader_election.cc:288] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f [CANDIDATE]: Term 1 election: Requested vote from peers 3a1a6c3187ba42caac4b7a4a6a30933d (127.7.111.65:41091), 33f78dce92df4e1eb43f72d8e07ab67b (127.7.111.66:46079)
I20260811 12:47:26.412206 20902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7699a7befe7482d89221fa6e9075838" candidate_uuid: "fca017e7ad634088988f027f77db0b8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33f78dce92df4e1eb43f72d8e07ab67b"
I20260811 12:47:26.412458 20902 raft_consensus.cc:3037] T a7699a7befe7482d89221fa6e9075838 P 33f78dce92df4e1eb43f72d8e07ab67b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:26.416507 20902 raft_consensus.cc:2454] T a7699a7befe7482d89221fa6e9075838 P 33f78dce92df4e1eb43f72d8e07ab67b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fca017e7ad634088988f027f77db0b8f in term 1.
I20260811 12:47:26.420248 21305 raft_consensus.cc:686] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f [term 1 LEADER]: Becoming Leader. State: Replica: fca017e7ad634088988f027f77db0b8f, State: Running, Role: LEADER
I20260811 12:47:26.420605 21305 consensus_queue.cc:237] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } }
I20260811 12:47:26.439906 20383 catalog_manager.cc:5495] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f reported cstate change: term changed from 0 to 1, leader changed from <none> to fca017e7ad634088988f027f77db0b8f (127.7.111.67). New cstate: current_term: 1 leader_uuid: "fca017e7ad634088988f027f77db0b8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fca017e7ad634088988f027f77db0b8f" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43503 } health_report { overall_health: HEALTHY } } }
I20260811 12:47:26.478806 21288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:26.472287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:53183 (local address 127.7.111.66:46079)
0811 12:47:26.472503 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:26.472510 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:26.473281 (+ 771us) server_negotiation.cc:407] Connection header received
0811 12:47:26.473520 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:26.473525 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:26.473601 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:26.473686 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:47:26.474326 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.475136 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.476725 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.477239 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.477369 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:26.477896 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:47:26.478065 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:47:26.478168 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:26.478327 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 12:47:26.478419 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 12:47:26.479305 21455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:26.472305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:46079 (local address 127.7.111.67:53183)
0811 12:47:26.473155 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:26.473173 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:26.473341 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:26.473721 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:26.473725 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:26.473744 (+ 19us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:47:26.474151 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:26.474162 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.475280 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.475287 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:26.476570 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:26.476585 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.477479 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.477486 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:26.477630 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:26.477699 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:47:26.478172 (+ 473us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:47:26.478187 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 12:47:26.478266 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 12:47:26.478425 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":435,"threads_started":1}
I20260811 12:47:26.492305 20902 raft_consensus.cc:1261] T a7699a7befe7482d89221fa6e9075838 P 33f78dce92df4e1eb43f72d8e07ab67b [term 1 FOLLOWER]: Refusing update from remote peer fca017e7ad634088988f027f77db0b8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:47:26.492954 20647 raft_consensus.cc:1261] T a7699a7befe7482d89221fa6e9075838 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 1 FOLLOWER]: Refusing update from remote peer fca017e7ad634088988f027f77db0b8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:47:26.493342 21448 consensus_queue.cc:1035] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "33f78dce92df4e1eb43f72d8e07ab67b" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:47:26.494043 21448 consensus_queue.cc:1035] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a1a6c3187ba42caac4b7a4a6a30933d" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:47:26.553057 21435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:26.537918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:44779 (local address 127.7.111.67:43503)
0811 12:47:26.538116 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:26.538122 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:26.546315 (+ 8193us) server_negotiation.cc:407] Connection header received
0811 12:47:26.546718 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:26.546725 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:26.546831 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:26.546906 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:47:26.547864 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.548765 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.550505 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.551222 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.551331 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:26.552063 (+ 732us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:47:26.552210 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:47:26.552286 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:26.552573 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 12:47:26.552679 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:47:26.553752 21461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:26.537568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:43503 (local address 127.7.111.66:44779)
0811 12:47:26.546183 (+ 8615us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:26.546213 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:26.546509 (+ 296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:26.547196 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:26.547201 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:26.547215 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:47:26.547658 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:26.547674 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.548952 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.548956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:26.550289 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:26.550311 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.551557 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.551569 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:26.551791 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:26.551847 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:47:26.552385 (+ 538us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:47:26.552396 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 12:47:26.553016 (+ 620us) client_negotiation.cc:240] Negotiation successful
0811 12:47:26.553185 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8428,"thread_start_us":8033,"threads_started":1}
I20260811 12:47:26.606660 20382 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:47:26.606925 20382 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:47:26.610088 20382 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9079ec9fcd5047f085fc64150f49664c: Sending DeleteTablet for 3 replicas of tablet a7699a7befe7482d89221fa6e9075838
I20260811 12:47:26.610989 20617 tablet_service.cc:1514] Processing DeleteTablet for tablet a7699a7befe7482d89221fa6e9075838 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:47:26 UTC) from {username='oryx', principal='oryx/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:36538
I20260811 12:47:26.611346 21394 tablet_replica.cc:330] T a7699a7befe7482d89221fa6e9075838 P 3a1a6c3187ba42caac4b7a4a6a30933d: stopping tablet replica
I20260811 12:47:26.611534 21394 raft_consensus.cc:2229] T a7699a7befe7482d89221fa6e9075838 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:47:26.611608 21139 tablet_service.cc:1514] Processing DeleteTablet for tablet a7699a7befe7482d89221fa6e9075838 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:47:26 UTC) from {username='oryx', principal='oryx/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:33280
I20260811 12:47:26.611764 21394 raft_consensus.cc:2258] T a7699a7befe7482d89221fa6e9075838 P 3a1a6c3187ba42caac4b7a4a6a30933d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:47:26.611953 21395 tablet_replica.cc:330] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f: stopping tablet replica
I20260811 12:47:26.612126 21395 raft_consensus.cc:2229] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:47:26.612496 21395 raft_consensus.cc:2258] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:47:26.613166 21394 ts_tablet_manager.cc:1905] T a7699a7befe7482d89221fa6e9075838 P 3a1a6c3187ba42caac4b7a4a6a30933d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:47:26.613725 21395 ts_tablet_manager.cc:1905] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:47:26.618237 20862 tablet_service.cc:1514] Processing DeleteTablet for tablet a7699a7befe7482d89221fa6e9075838 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:47:26 UTC) from {username='oryx', principal='oryx/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:33648
I20260811 12:47:26.618556 21396 tablet_replica.cc:330] T a7699a7befe7482d89221fa6e9075838 P 33f78dce92df4e1eb43f72d8e07ab67b: stopping tablet replica
I20260811 12:47:26.618731 21396 raft_consensus.cc:2229] T a7699a7befe7482d89221fa6e9075838 P 33f78dce92df4e1eb43f72d8e07ab67b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:47:26.618947 21396 raft_consensus.cc:2258] T a7699a7befe7482d89221fa6e9075838 P 33f78dce92df4e1eb43f72d8e07ab67b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:47:26.620398 21396 ts_tablet_manager.cc:1905] T a7699a7befe7482d89221fa6e9075838 P 33f78dce92df4e1eb43f72d8e07ab67b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:47:26.620396 21395 ts_tablet_manager.cc:1918] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:47:26.620544 21395 log.cc:1196] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/a7699a7befe7482d89221fa6e9075838
I20260811 12:47:26.621028 21395 ts_tablet_manager.cc:1939] T a7699a7befe7482d89221fa6e9075838 P fca017e7ad634088988f027f77db0b8f: Deleting consensus metadata
I20260811 12:47:26.622339 20364 catalog_manager.cc:4841] TS fca017e7ad634088988f027f77db0b8f (127.7.111.67:43503): tablet a7699a7befe7482d89221fa6e9075838 (table test-table [id=bb7ae0bc99ab4066a23104cbcf017634]) successfully deleted
I20260811 12:47:26.626760 21396 ts_tablet_manager.cc:1918] T a7699a7befe7482d89221fa6e9075838 P 33f78dce92df4e1eb43f72d8e07ab67b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:47:26.626892 21396 log.cc:1196] T a7699a7befe7482d89221fa6e9075838 P 33f78dce92df4e1eb43f72d8e07ab67b: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/a7699a7befe7482d89221fa6e9075838
I20260811 12:47:26.627543 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 20509
I20260811 12:47:26.632181 21394 ts_tablet_manager.cc:1918] T a7699a7befe7482d89221fa6e9075838 P 3a1a6c3187ba42caac4b7a4a6a30933d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:47:26.632462 21394 log.cc:1196] T a7699a7befe7482d89221fa6e9075838 P 3a1a6c3187ba42caac4b7a4a6a30933d: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/a7699a7befe7482d89221fa6e9075838
I20260811 12:47:26.633010 21394 ts_tablet_manager.cc:1939] T a7699a7befe7482d89221fa6e9075838 P 3a1a6c3187ba42caac4b7a4a6a30933d: Deleting consensus metadata
I20260811 12:47:26.633239 21396 ts_tablet_manager.cc:1939] T a7699a7befe7482d89221fa6e9075838 P 33f78dce92df4e1eb43f72d8e07ab67b: Deleting consensus metadata
I20260811 12:47:26.634439 20366 catalog_manager.cc:4841] TS 33f78dce92df4e1eb43f72d8e07ab67b (127.7.111.66:46079): tablet a7699a7befe7482d89221fa6e9075838 (table test-table [id=bb7ae0bc99ab4066a23104cbcf017634]) successfully deleted
I20260811 12:47:26.634439 20367 catalog_manager.cc:4841] TS 3a1a6c3187ba42caac4b7a4a6a30933d (127.7.111.65:41091): tablet a7699a7befe7482d89221fa6e9075838 (table test-table [id=bb7ae0bc99ab4066a23104cbcf017634]) successfully deleted
Aug 11 12:47:26 dist-test-slave-f0tn krb5kdc[20317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.111.65@KRBTEST.COM
I20260811 12:47:26.652069 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:26.629084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36548 (local address 127.7.111.65:41091)
0811 12:47:26.629316 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:26.629323 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:26.637472 (+ 8149us) server_negotiation.cc:407] Connection header received
0811 12:47:26.637724 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:26.637730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:26.637800 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:26.637882 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:26.639451 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.640230 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.641590 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.642116 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.642237 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:26.646397 (+ 4160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:26.646414 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:26.646430 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:26.646466 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:26.649332 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:26.650024 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:26.650031 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:26.650034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:26.650169 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:26.650534 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:26.650538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:26.650540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:26.650925 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:26.651250 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:26.651532 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 12:47:26.651773 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:47:26.652699 20726 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:26.791273 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 20509
I20260811 12:47:26.821867 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 20738
Aug 11 12:47:26 dist-test-slave-f0tn krb5kdc[20317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.111.66@KRBTEST.COM
I20260811 12:47:26.838160 21288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:26.822741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33680 (local address 127.7.111.66:46079)
0811 12:47:26.822941 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:26.822948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:26.823018 (+ 70us) server_negotiation.cc:407] Connection header received
0811 12:47:26.823219 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:26.823225 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:26.823333 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:26.823437 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:26.824795 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.825602 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.826882 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:26.827304 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:26.827417 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:26.832103 (+ 4686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:26.832121 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:26.832140 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:26.832190 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:26.835151 (+ 2961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:26.835928 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:26.835937 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:26.835939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:26.836076 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:26.836459 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:26.836463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:26.836465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:26.836876 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:26.837211 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:26.837535 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 12:47:26.837807 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:47:26.838898 20999 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:27.017650 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 20738
I20260811 12:47:27.049795 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 21015
Aug 11 12:47:27 dist-test-slave-f0tn krb5kdc[20317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.111.67@KRBTEST.COM
I20260811 12:47:27.065040 21435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:27.050605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33300 (local address 127.7.111.67:43503)
0811 12:47:27.050795 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:27.050800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:27.050842 (+ 42us) server_negotiation.cc:407] Connection header received
0811 12:47:27.051050 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:27.051054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:27.051121 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:27.051209 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:27.052516 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:27.053221 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:27.054513 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:27.054917 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:27.055014 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:27.059443 (+ 4429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:27.059461 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:27.059480 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:27.059523 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:27.062564 (+ 3041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:27.063254 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:27.063260 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:27.063263 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:27.063403 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:27.063703 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:27.063706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:27.063708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:27.064061 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:27.064328 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:27.064545 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 12:47:27.064722 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:47:27.065864 21250 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:27.192062 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 21015
I20260811 12:47:27.233008 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 20339
I20260811 12:47:27.234210 20448 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:27.352641 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 20339
2026-08-11T12:47:27Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7122 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:47:27 dist-test-slave-f0tn krb5kdc[21478](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:47:27 dist-test-slave-f0tn krb5kdc[21478](info): set up 2 sockets
Aug 11 12:47:27 dist-test-slave-f0tn krb5kdc[21478](info): commencing operation
krb5kdc: starting...
W20260811 12:47:28.603689 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:47:28 dist-test-slave-f0tn krb5kdc[21478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:47:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:47:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:47:28.876065 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:40011
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:34979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:40011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:47:29.210173 21499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:29.210465 21499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:29.210609 21499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:29.218030 21499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:47:29.218144 21499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:29.218204 21499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:29.218281 21499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:47:29.218339 21499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:47:29.228500 21499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:34979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:40011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:40011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:29.229882 21499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:29.231253 21499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:29.242997 21505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:29.243076 21510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:29.243036 21507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:29.244347 21499 server_base.cc:1034] running on GCE node
I20260811 12:47:29.245154 21499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:29.246763 21499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:29.247997 21499 hybrid_clock.cc:648] HybridClock initialized: now 1786452449247928 us; error 58 us; skew 500 ppm
I20260811 12:47:29.248382 21499 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T12:47:29Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1955 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:47:29 dist-test-slave-f0tn krb5kdc[21521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:47:29 dist-test-slave-f0tn krb5kdc[21521](info): set up 2 sockets
Aug 11 12:47:29 dist-test-slave-f0tn krb5kdc[21521](info): commencing operation
krb5kdc: starting...
W20260811 12:47:30.565905 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.012s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:47:30 dist-test-slave-f0tn krb5kdc[21521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:47:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:47:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:47:30.840581 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:38643
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:38643
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:47:31.172971 21539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:31.173312 21539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:31.173480 21539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:31.180734 21539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:47:31.180860 21539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:31.180943 21539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:31.181032 21539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:47:31.181094 21539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:47:31.191258 21539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:38643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:38643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:31.192615 21539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:31.193976 21539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:31.205595 21546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:31.205623 21549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:31.205700 21555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:31.206198 21539 server_base.cc:1034] running on GCE node
I20260811 12:47:31.206996 21539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:31.208446 21539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:31.209645 21539 hybrid_clock.cc:648] HybridClock initialized: now 1786452451209595 us; error 35 us; skew 500 ppm
I20260811 12:47:31.209982 21539 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:31 dist-test-slave-f0tn krb5kdc[21521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452451, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:31.216379 21539 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:47:31.217960 21539 webserver.cc:460] Webserver started at http://127.7.111.126:39313/ using document root <none> and password file <none>
I20260811 12:47:31.218665 21539 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:31.218789 21539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:31.219120 21539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:31.222247 21539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "374d4d03d56945bea5d0e79946afe214"
format_stamp: "Formatted at 2026-08-11 12:47:31 on dist-test-slave-f0tn"
server_key: "4780fedf2cb0dee6dfe8c3bb03cb8256"
server_key_iv: "a9f50147d5db156a90b854d306fb5bbf"
server_key_version: "encryptionkey@0"
I20260811 12:47:31.223027 21539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "374d4d03d56945bea5d0e79946afe214"
format_stamp: "Formatted at 2026-08-11 12:47:31 on dist-test-slave-f0tn"
server_key: "4780fedf2cb0dee6dfe8c3bb03cb8256"
server_key_iv: "a9f50147d5db156a90b854d306fb5bbf"
server_key_version: "encryptionkey@0"
I20260811 12:47:31.228083 21539 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 12:47:31.231460 21565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:31.232674 21539 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:47:31.232846 21539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "374d4d03d56945bea5d0e79946afe214"
format_stamp: "Formatted at 2026-08-11 12:47:31 on dist-test-slave-f0tn"
server_key: "4780fedf2cb0dee6dfe8c3bb03cb8256"
server_key_iv: "a9f50147d5db156a90b854d306fb5bbf"
server_key_version: "encryptionkey@0"
I20260811 12:47:31.233016 21539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:31.247953 21539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:31.251754 21539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:31.252008 21539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:31.269277 21539 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:38643
I20260811 12:47:31.269301 21653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:38643 every 8 connection(s)
I20260811 12:47:31.270659 21539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:47:31.274438 21654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:31.276356 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 21539
I20260811 12:47:31.276575 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:47:31.277227 7613 external_mini_cluster.cc:1428] Setting key 6daad4f5069af4ccf5c2e99129e1a87c
I20260811 12:47:31.286291 21654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214: Bootstrap starting.
I20260811 12:47:31.289988 21654 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:31.291301 21654 log.cc:824] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214: Log is configured to *not* fsync() on all Append() calls
Aug 11 12:47:31 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:47:31.294363 21654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214: No bootstrap required, opened a new log
I20260811 12:47:31.300081 21662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:31.279265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51798 (local address 127.7.111.126:38643)
0811 12:47:31.280124 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:31.280136 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:31.280177 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:47:31.281903 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:31.281926 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:31.282389 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:31.282740 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:31.284541 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:31.285866 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:31.287548 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:31.288235 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:31.288385 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:31.293071 (+ 4686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:31.293080 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:31.293094 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:31.293130 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:31.296133 (+ 3003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:31.296928 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:31.296934 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:31.296944 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:31.297102 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:31.297510 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:31.297514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:31.297515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:31.298048 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:31.298288 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:31.298692 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 12:47:31.299013 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":383,"threads_started":1}
I20260811 12:47:31.303418 21654 raft_consensus.cc:348] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374d4d03d56945bea5d0e79946afe214" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38643 } }
I20260811 12:47:31.303762 21654 raft_consensus.cc:374] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:31.303854 21654 raft_consensus.cc:729] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 374d4d03d56945bea5d0e79946afe214, State: Initialized, Role: FOLLOWER
I20260811 12:47:31.304411 21654 consensus_queue.cc:260] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374d4d03d56945bea5d0e79946afe214" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38643 } }
I20260811 12:47:31.304637 21654 raft_consensus.cc:388] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:47:31.304774 21654 raft_consensus.cc:482] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:47:31.304925 21654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:31.307098 21654 raft_consensus.cc:504] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374d4d03d56945bea5d0e79946afe214" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38643 } }
I20260811 12:47:31.307569 21654 leader_election.cc:302] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 374d4d03d56945bea5d0e79946afe214; no voters:
I20260811 12:47:31.308135 21654 leader_election.cc:288] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:47:31.308252 21666 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:47:31.309338 21666 raft_consensus.cc:686] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [term 1 LEADER]: Becoming Leader. State: Replica: 374d4d03d56945bea5d0e79946afe214, State: Running, Role: LEADER
I20260811 12:47:31.309882 21666 consensus_queue.cc:237] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374d4d03d56945bea5d0e79946afe214" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38643 } }
I20260811 12:47:31.310375 21654 sys_catalog.cc:564] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:47:31.312433 21669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 374d4d03d56945bea5d0e79946afe214. Latest consensus state: current_term: 1 leader_uuid: "374d4d03d56945bea5d0e79946afe214" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374d4d03d56945bea5d0e79946afe214" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38643 } } }
I20260811 12:47:31.312826 21669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:31.313426 21668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "374d4d03d56945bea5d0e79946afe214" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374d4d03d56945bea5d0e79946afe214" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38643 } } }
I20260811 12:47:31.313704 21668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:31.313795 21677 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:47:31.319831 21677 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:47:31.329463 21677 catalog_manager.cc:1347] Generated new cluster ID: 4d7578077abb480b83f9a049c6787465
I20260811 12:47:31.329604 21677 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:47:31.357057 21677 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:47:31.359010 21677 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:47:31.377120 21677 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214: Generated new TSK 0
I20260811 12:47:31.378028 21677 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.111.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
I20260811 12:47:31.442559 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:38643
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:47:31.776520 21699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:31.776801 21699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:31.776947 21699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:31.784591 21699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:31.784705 21699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:31.784873 21699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:47:31.794685 21699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:38643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:31.796010 21699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:31.797343 21699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:31.810240 21713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:31.810415 21712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:31.810434 21715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:31.811473 21699 server_base.cc:1034] running on GCE node
I20260811 12:47:31.812116 21699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:31.813093 21699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:31.814313 21699 hybrid_clock.cc:648] HybridClock initialized: now 1786452451814267 us; error 32 us; skew 500 ppm
I20260811 12:47:31.814646 21699 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:31 dist-test-slave-f0tn krb5kdc[21521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452451, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:31.817811 21699 init.cc:377] Logged in from keytab as kudu/127.7.111.65@KRBTEST.COM (short username kudu)
I20260811 12:47:31.819242 21699 webserver.cc:460] Webserver started at http://127.7.111.65:37473/ using document root <none> and password file <none>
I20260811 12:47:31.819936 21699 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:31.820055 21699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:31.820358 21699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:31.823380 21699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "cd4052370f2f4720b78b3797ec6fe812"
format_stamp: "Formatted at 2026-08-11 12:47:31 on dist-test-slave-f0tn"
server_key: "c4b857ca5c98911854f9b96d9565978f"
server_key_iv: "3017a4d7a73e6d7332c98b09019bb169"
server_key_version: "encryptionkey@0"
I20260811 12:47:31.824194 21699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "cd4052370f2f4720b78b3797ec6fe812"
format_stamp: "Formatted at 2026-08-11 12:47:31 on dist-test-slave-f0tn"
server_key: "c4b857ca5c98911854f9b96d9565978f"
server_key_iv: "3017a4d7a73e6d7332c98b09019bb169"
server_key_version: "encryptionkey@0"
I20260811 12:47:31.829756 21699 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 12:47:31.833324 21724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:31.834642 21699 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:47:31.834800 21699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "cd4052370f2f4720b78b3797ec6fe812"
format_stamp: "Formatted at 2026-08-11 12:47:31 on dist-test-slave-f0tn"
server_key: "c4b857ca5c98911854f9b96d9565978f"
server_key_iv: "3017a4d7a73e6d7332c98b09019bb169"
server_key_version: "encryptionkey@0"
I20260811 12:47:31.834982 21699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:31.861918 21699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:31.866253 21699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:31.866526 21699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:31.867470 21699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:31.868796 21699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:31.868875 21699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:31.869001 21699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:31.869056 21699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:31.899390 21699 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:44201
I20260811 12:47:31.899430 21895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:44201 every 8 connection(s)
I20260811 12:47:31.900631 21699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:47:31.909216 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 21699
I20260811 12:47:31.909463 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:47:31.910264 7613 external_mini_cluster.cc:1428] Setting key ee927de076b2bb327ed39347bf4fbda5
Aug 11 12:47:31 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452451, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:47:31.920609 21900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:31.903096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:47057 (local address 127.7.111.126:38643)
0811 12:47:31.903559 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:31.903565 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:31.904503 (+ 938us) server_negotiation.cc:407] Connection header received
0811 12:47:31.906280 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:31.906284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:31.906353 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:31.906447 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:31.907816 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:31.908521 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:31.910716 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:31.911269 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:31.911423 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:31.914867 (+ 3444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:31.914873 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:31.914877 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:31.914913 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:31.917758 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:31.918433 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:31.918438 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:31.918441 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:31.918586 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:31.918963 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:31.918967 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:31.918970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:31.919278 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:31.919419 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:31.920043 (+ 624us) server_negotiation.cc:299] Negotiation successful
0811 12:47:31.920278 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":215,"threads_started":1}
I20260811 12:47:31.921356 21902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:31.903658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.65:47057)
0811 12:47:31.904333 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:31.904376 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:31.906039 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:31.906621 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:31.906628 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:31.907124 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:31.907657 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:31.907674 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:31.908636 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:31.908640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:31.910566 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:31.910586 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:31.911503 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:31.911511 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:31.911766 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:31.912380 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:31.912402 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:31.914641 (+ 2239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:31.917878 (+ 3237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:31.917884 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:31.917896 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:31.918312 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:31.918684 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:31.918687 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:31.918689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:31.918862 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:31.919383 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:31.919388 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:31.919681 (+ 293us) client_negotiation.cc:769] Sending connection context
0811 12:47:31.919900 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 12:47:31.920266 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":256,"threads_started":1}
I20260811 12:47:31.922920 21898 heartbeater.cc:347] Connected to a master server at 127.7.111.126:38643
I20260811 12:47:31.923257 21898 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:31.924081 21898 heartbeater.cc:511] Master 127.7.111.126:38643 requested a full tablet report, sending...
I20260811 12:47:31.926172 21588 ts_manager.cc:194] Registered new tserver with Master: cd4052370f2f4720b78b3797ec6fe812 (127.7.111.65:44201)
I20260811 12:47:31.927595 21588 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:47057
WARNING: no policy specified for kudu/127.7.111.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
I20260811 12:47:31.968214 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:38643
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:47:32.301131 21905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:32.301468 21905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:32.301616 21905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:32.308856 21905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:32.308977 21905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:32.309131 21905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:47:32.318966 21905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:38643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:32.320274 21905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:32.321738 21905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:32.334753 21911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:32.334658 21912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:32.334728 21917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:32.335687 21905 server_base.cc:1034] running on GCE node
I20260811 12:47:32.336269 21905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:32.337193 21905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:32.338403 21905 hybrid_clock.cc:648] HybridClock initialized: now 1786452452338329 us; error 83 us; skew 500 ppm
I20260811 12:47:32.338716 21905 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:32 dist-test-slave-f0tn krb5kdc[21521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452452, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:32.341864 21905 init.cc:377] Logged in from keytab as kudu/127.7.111.66@KRBTEST.COM (short username kudu)
I20260811 12:47:32.343326 21905 webserver.cc:460] Webserver started at http://127.7.111.66:36569/ using document root <none> and password file <none>
I20260811 12:47:32.343990 21905 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:32.344086 21905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:32.344348 21905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:32.347249 21905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "47cbd572c1354e7688fdd6f88bae1c1c"
format_stamp: "Formatted at 2026-08-11 12:47:32 on dist-test-slave-f0tn"
server_key: "731c2b908e39abb70bffeccf1ef2dd5f"
server_key_iv: "93e29da6fdc005a6d0a694aa34dd9f82"
server_key_version: "encryptionkey@0"
I20260811 12:47:32.348003 21905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "47cbd572c1354e7688fdd6f88bae1c1c"
format_stamp: "Formatted at 2026-08-11 12:47:32 on dist-test-slave-f0tn"
server_key: "731c2b908e39abb70bffeccf1ef2dd5f"
server_key_iv: "93e29da6fdc005a6d0a694aa34dd9f82"
server_key_version: "encryptionkey@0"
I20260811 12:47:32.352911 21905 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 12:47:32.356244 21925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:32.357522 21905 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:47:32.357661 21905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "47cbd572c1354e7688fdd6f88bae1c1c"
format_stamp: "Formatted at 2026-08-11 12:47:32 on dist-test-slave-f0tn"
server_key: "731c2b908e39abb70bffeccf1ef2dd5f"
server_key_iv: "93e29da6fdc005a6d0a694aa34dd9f82"
server_key_version: "encryptionkey@0"
I20260811 12:47:32.357847 21905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:32.369853 21905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:32.373935 21905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:32.374197 21905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:32.375114 21905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:32.376442 21905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:32.376514 21905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:32.376626 21905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:32.376683 21905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:32.407791 21905 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:46859
I20260811 12:47:32.407828 22107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:46859 every 8 connection(s)
I20260811 12:47:32.409030 21905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:47:32.414233 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 21905
I20260811 12:47:32.414389 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:47:32.414820 7613 external_mini_cluster.cc:1428] Setting key 593601baa413819d21d5c6e534d8f775
Aug 11 12:47:32 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452452, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:47:32.430387 21900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:32.411652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:44793 (local address 127.7.111.126:38643)
0811 12:47:32.411844 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:32.411848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:32.413064 (+ 1216us) server_negotiation.cc:407] Connection header received
0811 12:47:32.414960 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:32.414967 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:32.415057 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:32.415176 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:32.416926 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:32.417798 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:32.419961 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:32.420395 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:32.420485 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:32.424228 (+ 3743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:32.424235 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:32.424239 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:32.424276 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:32.427238 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:32.427979 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:32.427984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:32.427987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:32.428141 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:32.428540 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:32.428544 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:32.428547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:32.428880 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:32.429010 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:32.429818 (+ 808us) server_negotiation.cc:299] Negotiation successful
0811 12:47:32.430043 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:47:32.431368 22113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:32.412204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.66:44793)
0811 12:47:32.412899 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:32.412942 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:32.414687 (+ 1745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:32.415387 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:32.415396 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:32.415984 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:32.416763 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:32.416784 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:32.417957 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:32.417962 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:32.419815 (+ 1853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:32.419832 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:32.420618 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:32.420625 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:32.420898 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:32.421599 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:32.421626 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:32.424010 (+ 2384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:32.427365 (+ 3355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:32.427373 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:32.427387 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:32.427866 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:32.428242 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:32.428246 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:32.428249 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:32.428443 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:32.428989 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:32.428996 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:32.429357 (+ 361us) client_negotiation.cc:769] Sending connection context
0811 12:47:32.429700 (+ 343us) client_negotiation.cc:240] Negotiation successful
0811 12:47:32.430144 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":255,"threads_started":1}
I20260811 12:47:32.432722 22110 heartbeater.cc:347] Connected to a master server at 127.7.111.126:38643
I20260811 12:47:32.433025 22110 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:32.433763 22110 heartbeater.cc:511] Master 127.7.111.126:38643 requested a full tablet report, sending...
I20260811 12:47:32.435181 21588 ts_manager.cc:194] Registered new tserver with Master: 47cbd572c1354e7688fdd6f88bae1c1c (127.7.111.66:46859)
I20260811 12:47:32.435981 21588 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:44793
WARNING: no policy specified for kudu/127.7.111.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
I20260811 12:47:32.474527 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:38643
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:47:32.822969 22118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:32.823258 22118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:32.823361 22118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:32.830605 22118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:32.830717 22118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:32.830885 22118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:47:32.841022 22118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:38643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:32.842386 22118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:32.843731 22118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:32.857188 22126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:32.857255 22130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:32.857304 22128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:32.858222 22118 server_base.cc:1034] running on GCE node
I20260811 12:47:32.858845 22118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:32.859840 22118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:32.861110 22118 hybrid_clock.cc:648] HybridClock initialized: now 1786452452861051 us; error 40 us; skew 500 ppm
I20260811 12:47:32.861479 22118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:32 dist-test-slave-f0tn krb5kdc[21521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452452, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:32.864718 22118 init.cc:377] Logged in from keytab as kudu/127.7.111.67@KRBTEST.COM (short username kudu)
I20260811 12:47:32.866226 22118 webserver.cc:460] Webserver started at http://127.7.111.67:44803/ using document root <none> and password file <none>
I20260811 12:47:32.866900 22118 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:32.867022 22118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:32.867327 22118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:32.870636 22118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "3e9fd0200b764ed1a5846fb01cfd71fb"
format_stamp: "Formatted at 2026-08-11 12:47:32 on dist-test-slave-f0tn"
server_key: "38786154cb3adedafff2e0746622549e"
server_key_iv: "3874274084fa5091b1a45bd340baadc7"
server_key_version: "encryptionkey@0"
I20260811 12:47:32.871450 22118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "3e9fd0200b764ed1a5846fb01cfd71fb"
format_stamp: "Formatted at 2026-08-11 12:47:32 on dist-test-slave-f0tn"
server_key: "38786154cb3adedafff2e0746622549e"
server_key_iv: "3874274084fa5091b1a45bd340baadc7"
server_key_version: "encryptionkey@0"
I20260811 12:47:32.876721 22118 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:47:32.880446 22146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:32.881891 22118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:47:32.882050 22118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "3e9fd0200b764ed1a5846fb01cfd71fb"
format_stamp: "Formatted at 2026-08-11 12:47:32 on dist-test-slave-f0tn"
server_key: "38786154cb3adedafff2e0746622549e"
server_key_iv: "3874274084fa5091b1a45bd340baadc7"
server_key_version: "encryptionkey@0"
I20260811 12:47:32.882251 22118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:32.907552 22118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:32.911743 22118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:32.912039 22118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:32.913028 22118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:32.914438 22118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:32.914511 22118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:32.914633 22118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:32.914683 22118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:32.930814 21898 heartbeater.cc:503] Master 127.7.111.126:38643 was elected leader, sending a full tablet report...
I20260811 12:47:32.946919 22118 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:43097
I20260811 12:47:32.946964 22318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:43097 every 8 connection(s)
I20260811 12:47:32.948318 22118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:47:32.955832 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 22118
I20260811 12:47:32.955986 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:47:32.956423 7613 external_mini_cluster.cc:1428] Setting key 12524b7ee110f4f0d5d8ca5e4c087eb4
Aug 11 12:47:32 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452452, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:47:32.967520 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:32.950791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:42637 (local address 127.7.111.126:38643)
0811 12:47:32.951212 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:32.951216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:32.952135 (+ 919us) server_negotiation.cc:407] Connection header received
0811 12:47:32.954038 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:32.954044 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:32.954125 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:32.954221 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:32.955862 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:32.956700 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:32.958537 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:32.958952 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:32.959064 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:32.962584 (+ 3520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:32.962590 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:32.962593 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:32.962625 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:32.964949 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:32.965691 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:32.965695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:32.965697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:32.965817 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:32.966156 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:32.966159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:32.966161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:32.966404 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:32.966511 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:32.967091 (+ 580us) server_negotiation.cc:299] Negotiation successful
0811 12:47:32.967265 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":216,"threads_started":1}
I20260811 12:47:32.968406 22323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:32.951276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.67:42637)
0811 12:47:32.951981 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:32.952021 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:32.953764 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:32.954464 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:32.954472 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:32.954981 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:32.955649 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:32.955677 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:32.956869 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:32.956875 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:32.958417 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:32.958431 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:32.959191 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:32.959201 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:32.959482 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:32.960095 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:32.960118 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:32.962407 (+ 2289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:32.965098 (+ 2691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:32.965105 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:32.965115 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:32.965586 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:32.965897 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:32.965901 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:32.965903 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:32.966078 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:32.966500 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:32.966506 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:32.966781 (+ 275us) client_negotiation.cc:769] Sending connection context
0811 12:47:32.967003 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 12:47:32.967359 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":268,"threads_started":1}
I20260811 12:47:32.969619 22319 heartbeater.cc:347] Connected to a master server at 127.7.111.126:38643
I20260811 12:47:32.969925 22319 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:32.970691 22319 heartbeater.cc:511] Master 127.7.111.126:38643 requested a full tablet report, sending...
I20260811 12:47:32.972080 21588 ts_manager.cc:194] Registered new tserver with Master: 3e9fd0200b764ed1a5846fb01cfd71fb (127.7.111.67:43097)
I20260811 12:47:32.972874 21588 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:42637
I20260811 12:47:32.983791 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.111.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.111.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.111.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.65.keytab.
Entry for principal oryx/127.7.111.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.65.keytab.
WARNING: no policy specified for oryx/127.7.111.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.111.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.111.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.66.keytab.
Entry for principal oryx/127.7.111.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.66.keytab.
WARNING: no policy specified for oryx/127.7.111.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.111.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.111.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.67.keytab.
Entry for principal oryx/127.7.111.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.67.keytab.
I20260811 12:47:33.158329 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 21699
Aug 11 12:47:33 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.65@KRBTEST.COM
I20260811 12:47:33.179078 22332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:33.159552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58946 (local address 127.7.111.65:44201)
0811 12:47:33.160818 (+ 1266us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:33.160828 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:33.160862 (+ 34us) server_negotiation.cc:407] Connection header received
0811 12:47:33.160941 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:33.160950 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:33.161246 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:33.161462 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:33.163140 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:33.164643 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:33.166527 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:33.167387 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:33.168072 (+ 685us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:33.172477 (+ 4405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:33.172492 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:33.172513 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:33.172556 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:33.176228 (+ 3672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:33.176918 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:33.176928 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:33.176932 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:33.177099 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:33.177435 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:33.177439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:33.177443 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:33.177882 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:33.178201 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:33.178378 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 12:47:33.178642 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1071,"thread_start_us":874,"threads_started":1}
I20260811 12:47:33.180498 21888 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:33.262815 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 21699
I20260811 12:47:33.283557 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 21905
Aug 11 12:47:33 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.66@KRBTEST.COM
I20260811 12:47:33.299486 22338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:33.284483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60434 (local address 127.7.111.66:46859)
0811 12:47:33.285039 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:33.285047 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:33.285075 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:47:33.285144 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:33.285153 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:33.285411 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:33.285557 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:33.286966 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:33.287981 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:33.289251 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:33.289888 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:33.290003 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:33.294073 (+ 4070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:33.294085 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:33.294101 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:33.294136 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:33.296771 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:33.297416 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:33.297426 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:33.297429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:33.297559 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:33.297899 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:33.297905 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:33.297909 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:33.298265 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:33.298552 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:33.298876 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 12:47:33.299129 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":261,"threads_started":1}
I20260811 12:47:33.300760 22102 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:33.382352 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 21905
I20260811 12:47:33.406334 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 22118
Aug 11 12:47:33 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.67@KRBTEST.COM
I20260811 12:47:33.422147 22343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:33.407313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56222 (local address 127.7.111.67:43097)
0811 12:47:33.407869 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:33.407876 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:33.407908 (+ 32us) server_negotiation.cc:407] Connection header received
0811 12:47:33.407971 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:33.407977 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:33.408191 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:33.408339 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:33.409739 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:33.410742 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:33.412038 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:33.412583 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:33.412707 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:33.416968 (+ 4261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:33.416979 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:33.416995 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:33.417028 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:33.419674 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:33.420296 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:33.420302 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:33.420305 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:33.420418 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:33.420713 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:33.420717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:33.420719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:33.421061 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:33.421298 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:33.421567 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 12:47:33.421821 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":259,"threads_started":1}
I20260811 12:47:33.423403 22312 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:33.505517 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 22118
I20260811 12:47:33.529691 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 21539
I20260811 12:47:33.530881 21645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:33.634936 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 21539
I20260811 12:47:33.665967 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:38643
--webserver_interface=127.7.111.126
--webserver_port=39313
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:38643
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:47:34.001667 22348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:34.001982 22348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:34.002146 22348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:34.009414 22348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:47:34.009536 22348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:34.009613 22348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:34.009693 22348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:47:34.009757 22348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:47:34.019726 22348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:38643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:38643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=39313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:34.021149 22348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:34.022573 22348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:34.034545 22354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:34.034595 22357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:34.034626 22360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:34.035392 22348 server_base.cc:1034] running on GCE node
I20260811 12:47:34.036154 22348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:34.037551 22348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:34.038786 22348 hybrid_clock.cc:648] HybridClock initialized: now 1786452454038731 us; error 39 us; skew 500 ppm
I20260811 12:47:34.039141 22348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:34 dist-test-slave-f0tn krb5kdc[21521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452454, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:34.042287 22348 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:47:34.043823 22348 webserver.cc:460] Webserver started at http://127.7.111.126:39313/ using document root <none> and password file <none>
I20260811 12:47:34.044523 22348 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:34.044656 22348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:34.050002 22348 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:47:34.052842 22368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:34.054195 22348 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:47:34.054355 22348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "374d4d03d56945bea5d0e79946afe214"
format_stamp: "Formatted at 2026-08-11 12:47:31 on dist-test-slave-f0tn"
server_key: "4780fedf2cb0dee6dfe8c3bb03cb8256"
server_key_iv: "a9f50147d5db156a90b854d306fb5bbf"
server_key_version: "encryptionkey@0"
I20260811 12:47:34.055332 22348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:34.080689 22348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:34.085106 22348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:34.085366 22348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:34.103484 22348 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:38643
I20260811 12:47:34.103511 22463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:38643 every 8 connection(s)
I20260811 12:47:34.105036 22348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:47:34.111884 22465 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:47:34.112833 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 22348
I20260811 12:47:34.113847 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:44201
--local_ip_for_outbound_sockets=127.7.111.65
--tserver_master_addrs=127.7.111.126:38643
--webserver_port=37473
--webserver_interface=127.7.111.65
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.111.65
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:47:34.116482 22465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214: Bootstrap starting.
I20260811 12:47:34.137933 22465 log.cc:824] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214: Log is configured to *not* fsync() on all Append() calls
I20260811 12:47:34.147173 22465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:47:34.147708 22465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214: Bootstrap complete.
I20260811 12:47:34.156739 22465 raft_consensus.cc:348] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374d4d03d56945bea5d0e79946afe214" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38643 } }
I20260811 12:47:34.157665 22465 raft_consensus.cc:729] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 374d4d03d56945bea5d0e79946afe214, State: Initialized, Role: FOLLOWER
I20260811 12:47:34.158264 22465 consensus_queue.cc:260] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374d4d03d56945bea5d0e79946afe214" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38643 } }
I20260811 12:47:34.158476 22465 raft_consensus.cc:388] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:47:34.158587 22465 raft_consensus.cc:482] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:47:34.158746 22465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [term 1 FOLLOWER]: Advancing to term 2
I20260811 12:47:34.162514 22465 raft_consensus.cc:504] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374d4d03d56945bea5d0e79946afe214" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38643 } }
I20260811 12:47:34.162984 22465 leader_election.cc:302] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 374d4d03d56945bea5d0e79946afe214; no voters:
I20260811 12:47:34.163592 22465 leader_election.cc:288] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 12:47:34.163744 22470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 12:47:34.165725 22470 raft_consensus.cc:686] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [term 2 LEADER]: Becoming Leader. State: Replica: 374d4d03d56945bea5d0e79946afe214, State: Running, Role: LEADER
I20260811 12:47:34.165776 22465 sys_catalog.cc:564] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:47:34.166313 22470 consensus_queue.cc:237] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374d4d03d56945bea5d0e79946afe214" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38643 } }
I20260811 12:47:34.171097 22473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 374d4d03d56945bea5d0e79946afe214. Latest consensus state: current_term: 2 leader_uuid: "374d4d03d56945bea5d0e79946afe214" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374d4d03d56945bea5d0e79946afe214" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38643 } } }
I20260811 12:47:34.171484 22473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:34.172050 22471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "374d4d03d56945bea5d0e79946afe214" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374d4d03d56945bea5d0e79946afe214" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38643 } } }
I20260811 12:47:34.172379 22471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:34.184880 22478 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:47:34.191174 22478 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:47:34.192740 22478 catalog_manager.cc:1259] Loaded cluster ID: 4d7578077abb480b83f9a049c6787465
I20260811 12:47:34.192900 22478 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:47:34.197414 22478 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:47:34.199038 22478 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 374d4d03d56945bea5d0e79946afe214: Loaded TSK: 0
I20260811 12:47:34.200646 22478 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 12:47:34.482456 22468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:34.482751 22468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:34.482887 22468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:34.490248 22468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:34.490372 22468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:34.490528 22468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:47:34.500276 22468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.65.keytab
--principal=oryx/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:44201
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=37473
--tserver_master_addrs=127.7.111.126:38643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:34.501679 22468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:34.503048 22468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:34.518349 22498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:34.523185 22497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:34.524956 22501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:34.525074 22468 server_base.cc:1034] running on GCE node
I20260811 12:47:34.525787 22468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:34.526804 22468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:34.528025 22468 hybrid_clock.cc:648] HybridClock initialized: now 1786452454527962 us; error 55 us; skew 500 ppm
I20260811 12:47:34.528343 22468 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:34 dist-test-slave-f0tn krb5kdc[21521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452454, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:34.531401 22468 init.cc:377] Logged in from keytab as oryx/127.7.111.65@KRBTEST.COM (short username oryx)
I20260811 12:47:34.532979 22468 webserver.cc:460] Webserver started at http://127.7.111.65:37473/ using document root <none> and password file <none>
I20260811 12:47:34.533720 22468 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:34.533844 22468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:34.539808 22468 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:47:34.543409 22509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:34.547065 22468 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 12:47:34.547240 22468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "cd4052370f2f4720b78b3797ec6fe812"
format_stamp: "Formatted at 2026-08-11 12:47:31 on dist-test-slave-f0tn"
server_key: "c4b857ca5c98911854f9b96d9565978f"
server_key_iv: "3017a4d7a73e6d7332c98b09019bb169"
server_key_version: "encryptionkey@0"
I20260811 12:47:34.548341 22468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:34.566591 22468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:34.570976 22468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:34.571204 22468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:34.572157 22468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:34.573510 22468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:34.573612 22468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:34.573704 22468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:34.573772 22468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:34.604612 22468 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:44201
I20260811 12:47:34.604681 22704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:44201 every 8 connection(s)
I20260811 12:47:34.606063 22468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:47:34.608806 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 22468
I20260811 12:47:34.609731 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:46859
--local_ip_for_outbound_sockets=127.7.111.66
--tserver_master_addrs=127.7.111.126:38643
--webserver_port=36569
--webserver_interface=127.7.111.66
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.111.66
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:47:34 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:34 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 12:47:34.949486 22712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:34.949771 22712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:34.949893 22712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:34.957150 22712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:34.957253 22712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:34.957448 22712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:47:34.967468 22712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.66.keytab
--principal=oryx/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:46859
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=36569
--tserver_master_addrs=127.7.111.126:38643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:34.968808 22712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:34.970176 22712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:34.985782 22729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:34.985837 22724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:34.986147 22732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:34.986785 22712 server_base.cc:1034] running on GCE node
I20260811 12:47:34.987365 22712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:34.988278 22712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:34.989468 22712 hybrid_clock.cc:648] HybridClock initialized: now 1786452454989399 us; error 71 us; skew 500 ppm
I20260811 12:47:34.989776 22712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:34 dist-test-slave-f0tn krb5kdc[21521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452454, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:34.993076 22712 init.cc:377] Logged in from keytab as oryx/127.7.111.66@KRBTEST.COM (short username oryx)
I20260811 12:47:34.994565 22712 webserver.cc:460] Webserver started at http://127.7.111.66:36569/ using document root <none> and password file <none>
I20260811 12:47:34.995302 22712 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:34.995428 22712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:35.001116 22712 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:47:35.004063 22740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:35.005641 22712 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:47:35.005808 22712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "47cbd572c1354e7688fdd6f88bae1c1c"
format_stamp: "Formatted at 2026-08-11 12:47:32 on dist-test-slave-f0tn"
server_key: "731c2b908e39abb70bffeccf1ef2dd5f"
server_key_iv: "93e29da6fdc005a6d0a694aa34dd9f82"
server_key_version: "encryptionkey@0"
I20260811 12:47:35.006773 22712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:35.032984 22712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:35.037077 22712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:35.037329 22712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:35.038476 22712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:35.039839 22712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:35.039933 22712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:35.040035 22712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:35.040112 22712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:35.074838 22712 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:46859
I20260811 12:47:35.074893 22937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:46859 every 8 connection(s)
I20260811 12:47:35.076298 22712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:47:35.078445 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 22712
I20260811 12:47:35.079054 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:43097
--local_ip_for_outbound_sockets=127.7.111.67
--tserver_master_addrs=127.7.111.126:38643
--webserver_port=44803
--webserver_interface=127.7.111.67
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.111.67
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:47:35 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:35 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 12:47:35.598476 22946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:35.598835 22946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:35.598987 22946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:35.607126 22946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:35.607259 22946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:35.607439 22946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:47:35.619724 22946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/krb5kdc/oryx_127.7.111.67.keytab
--principal=oryx/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:43097
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=44803
--tserver_master_addrs=127.7.111.126:38643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:35.621073 22946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:35.622474 22946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:35.638510 22963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:35.638552 22962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:35.639642 22967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:35.639763 22946 server_base.cc:1034] running on GCE node
I20260811 12:47:35.640689 22946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:35.642158 22946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:35.643435 22946 hybrid_clock.cc:648] HybridClock initialized: now 1786452455643370 us; error 44 us; skew 500 ppm
I20260811 12:47:35.643908 22946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:35 dist-test-slave-f0tn krb5kdc[21521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452455, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:35.648659 22946 init.cc:377] Logged in from keytab as oryx/127.7.111.67@KRBTEST.COM (short username oryx)
I20260811 12:47:35.650919 22946 webserver.cc:460] Webserver started at http://127.7.111.67:44803/ using document root <none> and password file <none>
I20260811 12:47:35.652034 22946 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:35.652220 22946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:35.660950 22946 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 12:47:35.664908 22975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:35.666976 22946 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260811 12:47:35.667196 22946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "3e9fd0200b764ed1a5846fb01cfd71fb"
format_stamp: "Formatted at 2026-08-11 12:47:32 on dist-test-slave-f0tn"
server_key: "38786154cb3adedafff2e0746622549e"
server_key_iv: "3874274084fa5091b1a45bd340baadc7"
server_key_version: "encryptionkey@0"
I20260811 12:47:35.668800 22946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:35.694360 22946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:35.700870 22946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:35.701225 22946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:35.702744 22946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:35.704758 22946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:35.704907 22946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:35.705034 22946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:35.705116 22946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260811 12:47:35.736785 22709 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.116s user 0.006s sys 0.008s
I20260811 12:47:35.738590 22709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:34.609040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.65:41401)
0811 12:47:34.609899 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:34.609954 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:34.611641 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:34.615082 (+ 3441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:34.615094 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:34.615681 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:34.616196 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:34.616223 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:34.617809 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:34.617813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:34.619022 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:34.619036 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:34.620037 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:34.620045 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:34.620301 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:34.620922 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:34.620944 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:35.737676 (+1116732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":366,"threads_started":1}
W20260811 12:47:35.740114 22710 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:34.609040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:41401 (local address 127.7.111.126:38643)
0811 12:47:34.613939 (+ 4899us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:34.613950 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:34.613988 (+ 38us) server_negotiation.cc:407] Connection header received
0811 12:47:34.614277 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:34.614289 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:34.614584 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:34.614896 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:34.616386 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:34.617650 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:34.619159 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:34.619813 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:34.619984 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:35.739492 (+1119508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.65:41401: BlockingRecv error: recv got EOF from 127.7.111.65:41401 (error 108)
Metrics: {"server-negotiator.queue_time_us":4687,"thread_start_us":4466,"threads_started":1}
W20260811 12:47:35.741006 22705 heartbeater.cc:644] Failed to heartbeat to 127.7.111.126:38643 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.111.126:38643: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Aug 11 12:47:35 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:35 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 12:47:35.752588 22710 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:35.741987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:49935 (local address 127.7.111.126:38643)
0811 12:47:35.742142 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:35.742148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:35.742165 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:47:35.742221 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:35.742224 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:35.742281 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:35.742358 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:35.743305 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:35.744346 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:35.746212 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:35.746807 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:35.746923 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:35.752335 (+ 5412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.65:49935: BlockingRecv error: recv got EOF from 127.7.111.65:49935 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:47:35.752961 22709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:35.741728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.65:49935)
0811 12:47:35.741921 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:35.741940 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:35.742061 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:35.742410 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:35.742414 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:35.742868 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:35.743107 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:35.743121 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:35.744530 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:35.744536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:35.746029 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:35.746045 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:35.747166 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:35.747174 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:35.747480 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:35.748171 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:35.748195 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:35.752692 (+ 4497us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":68}
Aug 11 12:47:35 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:35 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:35.762804 22709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:35.753652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.65:40507)
0811 12:47:35.753941 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:35.753958 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:35.754053 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:35.754502 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:35.754505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:35.754774 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:35.754938 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:35.754946 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:35.756270 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:35.756274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:35.757199 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:35.757211 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:35.758112 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:35.758119 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:35.758312 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:35.758978 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:35.759002 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:35.762555 (+ 3553us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184}
W20260811 12:47:35.762969 22710 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:35.753947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:40507 (local address 127.7.111.126:38643)
0811 12:47:35.754090 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:35.754095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:35.754111 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:47:35.754158 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:35.754163 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:35.754262 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:35.754352 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:35.755084 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:35.756112 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:35.757326 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:35.757932 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:35.758681 (+ 749us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:35.762761 (+ 4080us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.65:40507: BlockingRecv error: recv got EOF from 127.7.111.65:40507 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 12:47:35.763240 22705 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 12:47:35.766697 22946 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:43097
I20260811 12:47:35.766719 23164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:43097 every 8 connection(s)
I20260811 12:47:35.768860 22946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:47:35.778234 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 22946
Aug 11 12:47:35 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:35 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:35.808925 23170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:35.779114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50712 (local address 127.7.111.126:38643)
0811 12:47:35.786282 (+ 7168us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:35.786287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:35.789677 (+ 3390us) server_negotiation.cc:407] Connection header received
0811 12:47:35.789841 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:35.789847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:35.789934 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:35.790059 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:35.791964 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:35.793293 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:35.795196 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:35.795943 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:35.796130 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:35.799726 (+ 3596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:35.799741 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:35.799773 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:35.799828 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:35.804311 (+ 4483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:35.805359 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:35.805370 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:35.805413 (+ 43us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:35.805689 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:35.806232 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:35.806237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:35.806241 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:35.806987 (+ 746us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:35.807357 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:35.807948 (+ 591us) server_negotiation.cc:299] Negotiation successful
0811 12:47:35.808431 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7038,"thread_start_us":7270,"threads_started":1}
W20260811 12:47:36.206668 22941 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.100s user 0.008s sys 0.014s
W20260811 12:47:36.208067 22940 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:35.079253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:36101 (local address 127.7.111.126:38643)
0811 12:47:35.093768 (+ 14515us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:35.093773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:35.094328 (+ 555us) server_negotiation.cc:407] Connection header received
0811 12:47:35.097550 (+ 3222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:35.097556 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:35.097639 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:35.097758 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:35.100421 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:35.101525 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:35.104398 (+ 2873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:35.105157 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:35.105311 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:36.207720 (+1102409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.66:36101: BlockingRecv error: recv got EOF from 127.7.111.66:36101 (error 108)
Metrics: {"server-negotiator.queue_time_us":14393,"thread_start_us":14274,"threads_started":1}
I20260811 12:47:36.208676 22941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:35.079965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.66:36101)
0811 12:47:35.094098 (+ 14133us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:35.094165 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:35.097039 (+ 2874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:35.098319 (+ 1280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:35.098334 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:35.099250 (+ 916us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:35.100197 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:35.100223 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:35.101692 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:35.101698 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:35.104160 (+ 2462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:35.104180 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:35.105766 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:35.105774 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:35.106109 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:35.107037 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:35.107075 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:36.207474 (+1100399us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":13720,"thread_start_us":17793,"threads_started":1}
W20260811 12:47:36.209863 22938 heartbeater.cc:644] Failed to heartbeat to 127.7.111.126:38643 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.111.126:38643: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Aug 11 12:47:36 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:36 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:36.223948 22941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:36.210709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.66:49891)
0811 12:47:36.210913 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:36.210936 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:36.211084 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:36.211608 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:36.211614 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:36.211989 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:36.212189 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:36.212198 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:36.213863 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:36.213867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:36.214876 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:36.214889 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:36.215869 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:36.215875 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:36.216007 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:36.219730 (+ 3723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:36.219750 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:36.223643 (+ 3893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 12:47:36.224306 22940 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:36.211046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:49891 (local address 127.7.111.126:38643)
0811 12:47:36.211244 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:36.211251 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:36.211275 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:47:36.211341 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:36.211347 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:36.211436 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:36.211549 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:36.212332 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:36.213708 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:36.215003 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:36.215687 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:36.215816 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:36.224019 (+ 8203us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.66:49891: BlockingRecv error: recv got EOF from 127.7.111.66:49891 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 12:47:36 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:36 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:36.234690 22941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:36.225202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.66:33197)
0811 12:47:36.225462 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:36.225481 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:36.225586 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:36.225990 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:36.225993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:36.226378 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:36.226544 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:36.226553 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:36.228124 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:36.228127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:36.229068 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:36.229080 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:36.230034 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:36.230038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:36.230187 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:36.230815 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:36.230833 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:36.234483 (+ 3650us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133}
W20260811 12:47:36.235013 22938 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 12:47:36.235107 22940 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:36.225494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:33197 (local address 127.7.111.126:38643)
0811 12:47:36.225665 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:36.225672 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:36.225691 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:47:36.225755 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:36.225761 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:36.225855 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:36.225969 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:36.226700 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:36.227982 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:36.229185 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:36.229898 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:36.230038 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:36.234785 (+ 4747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.66:33197: BlockingRecv error: recv got EOF from 127.7.111.66:33197 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 11 12:47:36 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:36 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:36.774261 23174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:36.763972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.65:49137)
0811 12:47:36.764667 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:36.764690 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:36.764835 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:36.765237 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:36.765240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:36.765747 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:36.765943 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:36.765954 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:36.767088 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:36.767094 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:36.768607 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:36.768625 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:36.769636 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:36.769643 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:36.769824 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:36.770423 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:36.770442 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:36.773932 (+ 3490us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":334,"threads_started":1}
W20260811 12:47:36.774590 23178 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:36.764033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:49137 (local address 127.7.111.126:38643)
0811 12:47:36.764896 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:36.764902 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:36.764923 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:47:36.764985 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:36.764990 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:36.765092 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:36.765204 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:36.766101 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:36.766952 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:36.768780 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:36.769443 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:36.769565 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:36.774183 (+ 4618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.65:49137: BlockingRecv error: recv got EOF from 127.7.111.65:49137 (error 108)
Metrics: {"server-negotiator.queue_time_us":736,"thread_start_us":629,"threads_started":1}
W20260811 12:47:36.792528 23169 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.008s user 0.009s sys 0.007s
W20260811 12:47:36.793877 22710 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:35.771736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:43897 (local address 127.7.111.126:38643)
0811 12:47:35.771929 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:35.771936 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:35.773374 (+ 1438us) server_negotiation.cc:407] Connection header received
0811 12:47:35.776337 (+ 2963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:35.776341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:35.776399 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:35.776475 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:35.778683 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:35.779624 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:35.782151 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:35.782885 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:35.782985 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:36.793639 (+1010654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.67:43897: BlockingRecv error: recv got EOF from 127.7.111.67:43897 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:47:36.794550 23169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:35.772191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.67:43897)
0811 12:47:35.773127 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:35.773239 (+ 112us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:35.776033 (+ 2794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:35.776767 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:35.776780 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:35.777605 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:35.778470 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:35.778496 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:35.779849 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:35.779856 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:35.781974 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:35.781992 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:35.783369 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:35.783377 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:35.783681 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:35.784482 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:35.784522 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:36.793456 (+1008934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":341,"threads_started":1}
W20260811 12:47:36.795499 23165 heartbeater.cc:644] Failed to heartbeat to 127.7.111.126:38643 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.111.126:38643: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Aug 11 12:47:36 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:36 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:36.804855 23169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:36.796138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.67:36911)
0811 12:47:36.796374 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:36.796399 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:36.796526 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:36.796879 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:36.796884 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:36.797267 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:36.797475 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:36.797486 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:36.798530 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:36.798535 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:36.799496 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:36.799509 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:36.800265 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:36.800272 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:36.800428 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:36.800971 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:36.800988 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:36.804533 (+ 3545us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107}
W20260811 12:47:36.805058 22710 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:36.796257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:36911 (local address 127.7.111.126:38643)
0811 12:47:36.796503 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:36.796508 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:36.796529 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:47:36.796651 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:36.796656 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:36.796738 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:36.796825 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:36.797613 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:36.798391 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:36.799633 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:36.800073 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:36.800190 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:36.804810 (+ 4620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.67:36911: BlockingRecv error: recv got EOF from 127.7.111.67:36911 (error 108)
Metrics: {"server-negotiator.queue_time_us":132}
Aug 11 12:47:36 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:36 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:36.814121 23169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:36.805754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.67:49515)
0811 12:47:36.805915 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:36.805935 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:36.806046 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:36.806415 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:36.806419 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:36.806760 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:36.806941 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:36.806950 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:36.807910 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:36.807913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:36.808858 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:36.808869 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:36.809589 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:36.809595 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:36.809738 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:36.810363 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:36.810384 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:36.813843 (+ 3459us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 12:47:36.814319 22710 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:36.805822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:49515 (local address 127.7.111.126:38643)
0811 12:47:36.805979 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:36.805984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:36.806003 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:47:36.806157 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:36.806162 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:36.806250 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:36.806347 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:36.807058 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:36.807804 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:36.808962 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:36.809398 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:36.809496 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:36.814089 (+ 4593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.67:49515: BlockingRecv error: recv got EOF from 127.7.111.67:49515 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 12:47:36.814481 23165 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 12:47:37 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:37 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:37.244834 23180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:37.235637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.66:39939)
0811 12:47:37.236182 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:37.236204 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:37.236322 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:37.236586 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:37.236589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:37.237024 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:37.237185 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:37.237194 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:37.238234 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:37.238239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:37.239242 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:37.239256 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:37.240008 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:37.240013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:37.240172 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:37.240729 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:37.240747 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:37.244499 (+ 3752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":309,"threads_started":1}
W20260811 12:47:37.244988 22710 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:37.235781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:39939 (local address 127.7.111.126:38643)
0811 12:47:37.235947 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:37.235951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:37.236271 (+ 320us) server_negotiation.cc:407] Connection header received
0811 12:47:37.236407 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:37.236411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:37.236474 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:37.236555 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:37.237300 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:37.238067 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:37.239363 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:37.239805 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:37.239894 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:37.244739 (+ 4845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.66:39939: BlockingRecv error: recv got EOF from 127.7.111.66:39939 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 12:47:37 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:37 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:37.783986 23181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:37.775377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.65:57815)
0811 12:47:37.775920 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:37.775942 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:37.776090 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:37.776432 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:37.776435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:37.776825 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:37.777015 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:37.777025 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:37.778039 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:37.778042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:37.778985 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:37.778998 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:37.779685 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:37.779689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:37.779829 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:37.780386 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:37.780405 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:37.783741 (+ 3336us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":306,"threads_started":1}
W20260811 12:47:37.784212 23183 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:37.775434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:57815 (local address 127.7.111.126:38643)
0811 12:47:37.776142 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:37.776148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:37.776167 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:47:37.776222 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:37.776227 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:37.776298 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:37.776386 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:37.777137 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:37.777940 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:37.779119 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:37.779555 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:37.779649 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:37.783951 (+ 4302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.65:57815: BlockingRecv error: recv got EOF from 127.7.111.65:57815 (error 108)
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":476,"threads_started":1}
Aug 11 12:47:37 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:37 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:37.823714 23184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:37.815035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.67:53771)
0811 12:47:37.815616 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:37.815632 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:37.815747 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:37.816031 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:37.816035 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:37.816486 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:37.816635 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:37.816645 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:37.817575 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:37.817578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:37.818543 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:37.818555 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:37.819192 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:37.819197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:37.819319 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:37.819862 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:37.819879 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:37.823369 (+ 3490us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":332,"threads_started":1}
W20260811 12:47:37.823813 23183 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:37.815126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:53771 (local address 127.7.111.126:38643)
0811 12:47:37.815313 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:37.815317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:37.815696 (+ 379us) server_negotiation.cc:407] Connection header received
0811 12:47:37.815844 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:37.815847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:37.815922 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:37.816000 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:37.816746 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:37.817474 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:37.818647 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:37.819043 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:37.819138 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:37.823588 (+ 4450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.67:53771: BlockingRecv error: recv got EOF from 127.7.111.67:53771 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 12:47:38 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:38 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:38.255474 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:38.245948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.66:51179)
0811 12:47:38.246468 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:38.246491 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:38.246638 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:38.246961 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:38.246966 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:38.247432 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:38.247621 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:38.247632 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:38.248648 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:38.248654 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:38.249685 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:38.249699 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:38.250408 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:38.250414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:38.250572 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:38.251185 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:38.251205 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:38.255131 (+ 3926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":283,"threads_started":1}
W20260811 12:47:38.255561 23183 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:38.246074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:51179 (local address 127.7.111.126:38643)
0811 12:47:38.246313 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:38.246319 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:38.246570 (+ 251us) server_negotiation.cc:407] Connection header received
0811 12:47:38.246770 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:38.246774 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:38.246838 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:38.246914 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:38.247782 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:38.248502 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:38.249821 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:38.250255 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:38.250363 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:38.255304 (+ 4941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.66:51179: BlockingRecv error: recv got EOF from 127.7.111.66:51179 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 12:47:38 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:38 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:38.793941 23186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:38.784969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.65:45843)
0811 12:47:38.785544 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:38.785566 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:38.785704 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:38.786140 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:38.786144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:38.786651 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:38.786845 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:38.786855 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:38.787881 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:38.787885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:38.788796 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:38.788808 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:38.789550 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:38.789555 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:38.789726 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:38.790274 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:38.790292 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:38.793617 (+ 3325us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":287,"threads_started":1}
W20260811 12:47:38.794117 23188 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:38.785130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:45843 (local address 127.7.111.126:38643)
0811 12:47:38.785846 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:38.785852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:38.785869 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:47:38.785911 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:38.785915 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:38.785985 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:38.786071 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:38.786997 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:38.787731 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:38.788944 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:38.789410 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:38.789523 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:38.793865 (+ 4342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.65:45843: BlockingRecv error: recv got EOF from 127.7.111.65:45843 (error 108)
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":466,"threads_started":1}
Aug 11 12:47:38 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:38 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:38.833673 23189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:38.824738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.67:49285)
0811 12:47:38.825310 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:38.825330 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:38.825477 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:38.825777 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:38.825780 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:38.826183 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:38.826379 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:38.826388 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:38.827348 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:38.827352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:38.828274 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:38.828286 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:38.828951 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:38.828957 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:38.829096 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:38.829692 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:38.829724 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:38.833358 (+ 3634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":331,"threads_started":1}
W20260811 12:47:38.833822 23188 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:38.824837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:49285 (local address 127.7.111.126:38643)
0811 12:47:38.825024 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:38.825028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:38.825398 (+ 370us) server_negotiation.cc:407] Connection header received
0811 12:47:38.825588 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:38.825592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:38.825652 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:38.825745 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:38.826507 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:38.827227 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:38.828381 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:38.828785 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:38.828871 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:38.833598 (+ 4727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.67:49285: BlockingRecv error: recv got EOF from 127.7.111.67:49285 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 12:47:39 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:39 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:39.265918 23190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:39.256468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.66:38051)
0811 12:47:39.257038 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:39.257059 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:39.257178 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:39.257589 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:39.257594 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:39.258073 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:39.258279 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:39.258290 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:39.259384 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:39.259388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:39.260378 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:39.260392 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:39.261118 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:39.261123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:39.261299 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:39.261883 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:39.261902 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:39.265655 (+ 3753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":309,"threads_started":1}
W20260811 12:47:39.266163 23188 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:39.256574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:38051 (local address 127.7.111.126:38643)
0811 12:47:39.256761 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:39.256765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:39.257146 (+ 381us) server_negotiation.cc:407] Connection header received
0811 12:47:39.257291 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:39.257296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:39.257430 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:39.257526 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:39.258427 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:39.259252 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:39.260519 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:39.260971 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:39.261081 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:39.265863 (+ 4782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.66:38051: BlockingRecv error: recv got EOF from 127.7.111.66:38051 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 12:47:39 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:39 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:39.803889 23194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:39.794982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.65:57935)
0811 12:47:39.795533 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:39.795550 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:39.795661 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:39.796071 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:39.796075 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:39.796487 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:39.796699 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:39.796710 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:39.797971 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:39.797975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:39.798986 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:39.799000 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:39.799643 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:39.799647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:39.799784 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:39.800350 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:39.800367 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:39.803628 (+ 3261us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":275,"threads_started":1}
W20260811 12:47:39.804062 23195 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:39.795106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:57935 (local address 127.7.111.126:38643)
0811 12:47:39.795574 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:39.795580 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:39.795605 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:47:39.795808 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:39.795813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:39.795903 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:39.796003 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:39.796852 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:39.797832 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:39.799117 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:39.799535 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:39.799626 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:39.803835 (+ 4209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.65:57935: BlockingRecv error: recv got EOF from 127.7.111.65:57935 (error 108)
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":251,"threads_started":1}
Aug 11 12:47:39 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:39 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:39.843348 23196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:39.834623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.67:47411)
0811 12:47:39.835214 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:39.835231 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:39.835359 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:39.835694 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:39.835709 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:39.836081 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:39.836258 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:39.836268 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:39.837236 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:39.837239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:39.838198 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:39.838211 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:39.838814 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:39.838818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:39.838930 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:39.839535 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:39.839553 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:39.843006 (+ 3453us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":364,"threads_started":1}
W20260811 12:47:39.843444 23195 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:39.834673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:47411 (local address 127.7.111.126:38643)
0811 12:47:39.834888 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:39.834893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:39.835314 (+ 421us) server_negotiation.cc:407] Connection header received
0811 12:47:39.835452 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:39.835456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:39.835521 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:39.835598 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:39.836379 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:39.837126 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:39.838304 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:39.838709 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:39.838799 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:39.843235 (+ 4436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.67:47411: BlockingRecv error: recv got EOF from 127.7.111.67:47411 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
Aug 11 12:47:40 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:40 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:40.275862 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:40.266935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.66:32963)
0811 12:47:40.267489 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:40.267510 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:40.267623 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:40.267972 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:40.267976 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:40.268443 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:40.268625 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:40.268639 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:40.269979 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:40.269983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:40.271001 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:40.271015 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:40.271632 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:40.271635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:40.271768 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:40.272332 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:40.272350 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:40.275569 (+ 3219us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":312,"threads_started":1}
W20260811 12:47:40.276031 23195 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:40.266982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:32963 (local address 127.7.111.126:38643)
0811 12:47:40.267137 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:40.267141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:40.267578 (+ 437us) server_negotiation.cc:407] Connection header received
0811 12:47:40.267734 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:40.267740 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:40.267843 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:40.267952 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:40.268775 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:40.269864 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:40.271114 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:40.271524 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:40.271615 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:40.275811 (+ 4196us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.66:32963: BlockingRecv error: recv got EOF from 127.7.111.66:32963 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 12:47:40 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:40 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:40.813592 23198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:40.804813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.65:58657)
0811 12:47:40.805355 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:40.805371 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:40.805518 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:40.805925 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:40.805928 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:40.806391 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:40.806544 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:40.806553 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:40.807573 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:40.807576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:40.808509 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:40.808521 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:40.809204 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:40.809211 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:40.809352 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:40.809930 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:40.809949 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:40.813321 (+ 3372us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":281,"threads_started":1}
W20260811 12:47:40.813990 23199 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:40.804912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:58657 (local address 127.7.111.126:38643)
0811 12:47:40.805651 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:40.805655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:40.805672 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:47:40.805732 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:40.805735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:40.805806 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:40.805895 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:40.806680 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:40.807443 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:40.808614 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:40.809036 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:40.809149 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:40.813591 (+ 4442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.65:58657: BlockingRecv error: recv got EOF from 127.7.111.65:58657 (error 108)
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":520,"threads_started":1}
Aug 11 12:47:40 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:40 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:40.853783 23201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:40.844275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.67:33009)
0811 12:47:40.844800 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:40.844817 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:40.844938 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:40.845241 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:40.845245 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:40.845715 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:40.845893 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:40.845903 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:40.846897 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:40.846901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:40.848080 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:40.848096 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:40.848816 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:40.848820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:40.848959 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:40.849536 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:40.849567 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:40.853416 (+ 3849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":279,"threads_started":1}
W20260811 12:47:40.853874 23199 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:40.844406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:33009 (local address 127.7.111.126:38643)
0811 12:47:40.844652 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:40.844657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:40.844905 (+ 248us) server_negotiation.cc:407] Connection header received
0811 12:47:40.845043 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:40.845047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:40.845112 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:40.845190 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:40.846018 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:40.846775 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:40.848228 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:40.848656 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:40.848746 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:40.853639 (+ 4893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.67:33009: BlockingRecv error: recv got EOF from 127.7.111.67:33009 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 11 12:47:41 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:41 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:41.290578 23202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:41.276789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.66:51443)
0811 12:47:41.277499 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:41.277521 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:41.277650 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:41.277986 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:41.277992 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:41.278426 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:41.278586 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:41.278596 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:41.279594 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:41.279599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:41.280543 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:41.280556 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:41.281265 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:41.281271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:41.281471 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:41.286273 (+ 4802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:41.286295 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:41.290331 (+ 4036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":286,"threads_started":1}
W20260811 12:47:41.290762 23199 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:41.276984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:51443 (local address 127.7.111.126:38643)
0811 12:47:41.277203 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:41.277209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:41.277600 (+ 391us) server_negotiation.cc:407] Connection header received
0811 12:47:41.277770 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:41.277776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:41.277850 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:41.277930 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:41.278736 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:41.279454 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:41.280682 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:41.281119 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:41.281246 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:41.290541 (+ 9295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.66:51443: BlockingRecv error: recv got EOF from 127.7.111.66:51443 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 12:47:41 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:41 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:41.824070 23205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:41.814660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.65:34593)
0811 12:47:41.815473 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:41.815503 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:41.815658 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:41.816037 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:41.816043 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:41.816521 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:41.816700 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:41.816711 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:41.817856 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:41.817862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:41.818862 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:41.818876 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:41.819649 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:41.819656 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:41.819846 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:41.820433 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:41.820452 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:41.823759 (+ 3307us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":559,"threads_started":1}
W20260811 12:47:41.824340 23206 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:41.814774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:34593 (local address 127.7.111.126:38643)
0811 12:47:41.815297 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:41.815303 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:41.815601 (+ 298us) server_negotiation.cc:407] Connection header received
0811 12:47:41.815789 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:41.815796 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:41.815876 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:41.815976 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:41.816865 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:41.817706 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:41.819028 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:41.819475 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:41.819604 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:41.823965 (+ 4361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.65:34593: BlockingRecv error: recv got EOF from 127.7.111.65:34593 (error 108)
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":247,"threads_started":1}
Aug 11 12:47:41 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:41 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:41.863533 23207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:41.854654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.67:60955)
0811 12:47:41.855157 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:41.855181 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:41.855296 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:41.855571 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:41.855573 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:41.856015 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:41.856198 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:41.856209 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:41.857131 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:41.857134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:41.858102 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:41.858115 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:41.858827 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:41.858832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:41.858980 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:41.859589 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:41.859609 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:41.863223 (+ 3614us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":251,"threads_started":1}
W20260811 12:47:41.863627 23206 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:41.854728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:60955 (local address 127.7.111.126:38643)
0811 12:47:41.854887 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:41.854892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:41.855244 (+ 352us) server_negotiation.cc:407] Connection header received
0811 12:47:41.855407 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:41.855411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:41.855472 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:41.855543 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:41.856309 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:41.857036 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:41.858228 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:41.858706 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:41.858797 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:41.863399 (+ 4602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.67:60955: BlockingRecv error: recv got EOF from 127.7.111.67:60955 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 12:47:42 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:42 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:42.300899 23209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:42.291755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.66:38605)
0811 12:47:42.292333 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:42.292359 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:42.292482 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:42.292792 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:42.292796 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:42.293201 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:42.293412 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:42.293421 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:42.294455 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:42.294459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:42.295526 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:42.295543 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:42.296328 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:42.296334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:42.296469 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:42.297049 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:42.297066 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:42.300655 (+ 3589us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":340,"threads_started":1}
W20260811 12:47:42.301220 23206 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:42.291893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:38605 (local address 127.7.111.126:38643)
0811 12:47:42.292055 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:42.292060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:42.292429 (+ 369us) server_negotiation.cc:407] Connection header received
0811 12:47:42.292574 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:42.292578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:42.292657 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:42.292756 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:42.293531 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:42.294289 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:42.295676 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:42.296180 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:42.296278 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:42.300875 (+ 4597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.66:38605: BlockingRecv error: recv got EOF from 127.7.111.66:38605 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 12:47:42 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:42 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:42.833823 23211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:42.824942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.65:49491)
0811 12:47:42.825557 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:42.825575 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:42.825684 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:42.826065 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:42.826068 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:42.826492 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:42.826672 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:42.826681 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:42.827614 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:42.827617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:42.828576 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:42.828588 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:42.829225 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:42.829228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:42.829334 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:42.829883 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:42.829900 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:42.833568 (+ 3668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":257,"threads_started":1}
W20260811 12:47:42.834023 23210 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:42.825025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:49491 (local address 127.7.111.126:38643)
0811 12:47:42.825802 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:42.825807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:42.825822 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:47:42.825865 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:42.825869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:42.825941 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:42.826027 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:42.826778 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:42.827514 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:42.828696 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:42.829104 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:42.829192 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:42.833774 (+ 4582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.65:49491: BlockingRecv error: recv got EOF from 127.7.111.65:49491 (error 108)
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":577,"threads_started":1}
Aug 11 12:47:42 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:42 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:42.873859 23215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:42.864482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.67:51693)
0811 12:47:42.865025 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:42.865043 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:42.865170 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:42.865503 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:42.865506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:42.865970 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:42.866174 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:42.866185 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:42.867445 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:42.867449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:42.868680 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:42.868698 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:42.869484 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:42.869489 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:42.869622 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:42.870178 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:42.870196 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:42.873620 (+ 3424us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":264,"threads_started":1}
W20260811 12:47:42.874022 23210 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:42.864578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:51693 (local address 127.7.111.126:38643)
0811 12:47:42.864741 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:42.864746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:42.865112 (+ 366us) server_negotiation.cc:407] Connection header received
0811 12:47:42.865289 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:42.865293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:42.865358 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:42.865465 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:42.866368 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:42.867287 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:42.868829 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:42.869302 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:42.869440 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:42.873821 (+ 4381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.67:51693: BlockingRecv error: recv got EOF from 127.7.111.67:51693 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 12:47:43 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:43 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:43.311180 23216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:43.301996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.66:38423)
0811 12:47:43.302601 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:43.302622 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:43.302754 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:43.303122 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:43.303127 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:43.303559 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:43.303763 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:43.303774 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:43.304884 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:43.304888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:43.305929 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:43.305944 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:43.306658 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:43.306664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:43.306841 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:43.307389 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:43.307406 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:43.310815 (+ 3409us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":265,"threads_started":1}
W20260811 12:47:43.311334 23210 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:43.302053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:38423 (local address 127.7.111.126:38643)
0811 12:47:43.302312 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:43.302321 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:43.302719 (+ 398us) server_negotiation.cc:407] Connection header received
0811 12:47:43.302893 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:43.302897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:43.302979 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:43.303069 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:43.303917 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:43.304735 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:43.306076 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:43.306508 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:43.306616 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:43.311048 (+ 4432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.66:38423: BlockingRecv error: recv got EOF from 127.7.111.66:38423 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 11 12:47:43 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:43 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:43.843829 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:43.834716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.65:57265)
0811 12:47:43.835277 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:43.835293 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:43.835405 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:43.835714 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:43.835716 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:43.836133 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:43.836325 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:43.836334 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:43.837462 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:43.837467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:43.838421 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:43.838434 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:43.839126 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:43.839131 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:43.839259 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:43.839842 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:43.839858 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:43.843570 (+ 3712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":286,"threads_started":1}
W20260811 12:47:43.843977 23218 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:43.834838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:57265 (local address 127.7.111.126:38643)
0811 12:47:43.835342 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:43.835346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:43.835363 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:47:43.835515 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:43.835520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:43.835584 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:43.835676 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:43.836502 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:43.837300 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:43.838552 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:43.838977 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:43.839071 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:43.843761 (+ 4690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.65:57265: BlockingRecv error: recv got EOF from 127.7.111.65:57265 (error 108)
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":301,"threads_started":1}
Aug 11 12:47:43 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:43 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:43.883970 23219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:43.874911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.67:33975)
0811 12:47:43.875503 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:43.875534 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:43.875676 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:43.876051 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:43.876056 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:43.876494 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:43.876740 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:43.876756 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:43.877914 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:43.877919 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:43.878876 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:43.878889 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:43.879616 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:43.879622 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:43.879790 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:43.880331 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:43.880350 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:43.883611 (+ 3261us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":259,"threads_started":1}
W20260811 12:47:43.884048 23218 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:43.875045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:33975 (local address 127.7.111.126:38643)
0811 12:47:43.875234 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:43.875241 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:43.875623 (+ 382us) server_negotiation.cc:407] Connection header received
0811 12:47:43.875800 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:43.875806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:43.875911 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:43.875996 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:43.876917 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:43.877765 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:43.879023 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:43.879459 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:43.879566 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:43.883822 (+ 4256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.67:33975: BlockingRecv error: recv got EOF from 127.7.111.67:33975 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 12:47:44 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:44 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:44.321190 23220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:44.312279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.66:59845)
0811 12:47:44.312800 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:44.312822 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:44.312942 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:44.313223 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:44.313226 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:44.313704 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:44.313883 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:44.313893 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:44.315024 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:44.315028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:44.316049 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:44.316063 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:44.316881 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:44.316887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:44.317034 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:44.317685 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:44.317705 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:44.320941 (+ 3236us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":257,"threads_started":1}
W20260811 12:47:44.321365 23218 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:44.312329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:59845 (local address 127.7.111.126:38643)
0811 12:47:44.312489 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:44.312493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:44.312887 (+ 394us) server_negotiation.cc:407] Connection header received
0811 12:47:44.313026 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:44.313030 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:44.313107 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:44.313185 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:44.314007 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:44.314902 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:44.316188 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:44.316678 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:44.316813 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:44.321136 (+ 4323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.66:59845: BlockingRecv error: recv got EOF from 127.7.111.66:59845 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 12:47:44 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:44 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:44.854223 23221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:44.844794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.65:42891)
0811 12:47:44.845351 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:44.845367 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:44.845525 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:44.845906 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:44.845908 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:44.846348 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:44.846508 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:44.846518 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:44.847526 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:44.847530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:44.848475 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:44.848485 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:44.849294 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:44.849300 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:44.849495 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:44.850076 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:44.850094 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:44.853972 (+ 3878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":283,"threads_started":1}
W20260811 12:47:44.854404 23222 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:44.844898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:42891 (local address 127.7.111.126:38643)
0811 12:47:44.845647 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:44.845651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:44.845668 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:47:44.845717 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:44.845720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:44.845787 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:44.845874 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:44.846652 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:44.847423 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:44.848605 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:44.849105 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:44.849244 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:44.854151 (+ 4907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.65:42891: BlockingRecv error: recv got EOF from 127.7.111.65:42891 (error 108)
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":522,"threads_started":1}
Aug 11 12:47:44 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:44 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:44.893721 23226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:44.884875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.67:35863)
0811 12:47:44.885477 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:44.885493 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:44.885641 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:44.885970 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:44.885974 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:44.886401 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:44.886600 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:44.886611 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:44.887639 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:44.887645 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:44.888586 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:44.888598 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:44.889331 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:44.889338 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:44.889528 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:44.890111 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:44.890129 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:44.893462 (+ 3333us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":326,"threads_started":1}
W20260811 12:47:44.893834 23222 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:44.884988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:35863 (local address 127.7.111.126:38643)
0811 12:47:44.885217 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:44.885224 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:44.885577 (+ 353us) server_negotiation.cc:407] Connection header received
0811 12:47:44.885768 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:44.885773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:44.885836 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:44.885914 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:44.886750 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:44.887496 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:44.888732 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:44.889151 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:44.889260 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:44.893648 (+ 4388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.67:35863: BlockingRecv error: recv got EOF from 127.7.111.67:35863 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 11 12:47:45 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:45 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:45.331110 23227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:45.322198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.66:45693)
0811 12:47:45.322674 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:45.322697 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:45.322815 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:45.323113 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:45.323116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:45.323546 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:45.323702 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:45.323713 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:45.325001 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:45.325007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:45.326003 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:45.326020 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:45.326740 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:45.326746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:45.326925 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:45.327469 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:45.327487 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:45.330875 (+ 3388us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":238,"threads_started":1}
W20260811 12:47:45.331349 23222 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:45.322259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:45693 (local address 127.7.111.126:38643)
0811 12:47:45.322492 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:45.322498 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:45.322802 (+ 304us) server_negotiation.cc:407] Connection header received
0811 12:47:45.322902 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:45.322906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:45.322971 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:45.323080 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:45.323847 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:45.324856 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:45.326151 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:45.326584 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:45.326694 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:45.331104 (+ 4410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.66:45693: BlockingRecv error: recv got EOF from 127.7.111.66:45693 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 12:47:45 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:45 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:45.863620 23228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:45.855075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.65:40891)
0811 12:47:45.855596 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:45.855613 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:45.855734 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:45.856008 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:45.856011 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:45.856392 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:45.856563 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:45.856572 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:45.857602 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:45.857607 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:45.858685 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:45.858699 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:45.859490 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:45.859495 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:45.859618 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:45.860222 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:45.860243 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:45.863376 (+ 3133us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":265,"threads_started":1}
W20260811 12:47:45.863768 23229 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:45.855152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:40891 (local address 127.7.111.126:38643)
0811 12:47:45.855620 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:45.855624 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:45.855665 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:47:45.855821 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:45.855825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:45.855895 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:45.855978 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:45.856677 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:45.857439 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:45.858806 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:45.859347 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:45.859439 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:45.863557 (+ 4118us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.65:40891: BlockingRecv error: recv got EOF from 127.7.111.65:40891 (error 108)
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":247,"threads_started":1}
Aug 11 12:47:45 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:45 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:45.903214 23230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:45.894581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.67:43633)
0811 12:47:45.895068 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:45.895083 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:45.895208 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:45.895525 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:45.895529 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:45.895952 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:45.896099 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:45.896107 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:45.897119 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:45.897123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:45.898105 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:45.898118 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:45.898795 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:45.898802 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:45.898970 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:45.899547 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:45.899565 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:45.902908 (+ 3343us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":253,"threads_started":1}
W20260811 12:47:45.903316 23229 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:45.894647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:43633 (local address 127.7.111.126:38643)
0811 12:47:45.894850 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:45.894857 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:45.895162 (+ 305us) server_negotiation.cc:407] Connection header received
0811 12:47:45.895328 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:45.895333 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:45.895397 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:45.895474 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:45.896255 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:45.896982 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:45.898243 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:45.898648 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:45.898754 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:45.903087 (+ 4333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.67:43633: BlockingRecv error: recv got EOF from 127.7.111.67:43633 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 12:47:46 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:46 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:46.345474 23233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:46.332100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.66:52763)
0811 12:47:46.332635 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:46.332656 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:46.332799 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:46.333154 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:46.333159 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:46.333634 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:46.333808 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:46.333818 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:46.334866 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:46.334870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:46.335875 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:46.335889 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:46.336579 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:46.336585 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:46.336752 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:46.341479 (+ 4727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:46.341506 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:46.345183 (+ 3677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":295,"threads_started":1}
W20260811 12:47:46.345782 23229 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:46.332203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:52763 (local address 127.7.111.126:38643)
0811 12:47:46.332439 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:46.332446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:46.332750 (+ 304us) server_negotiation.cc:407] Connection header received
0811 12:47:46.332930 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:46.332934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:46.333006 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:46.333094 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:46.333970 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:46.334711 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:46.336023 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:46.336428 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:46.336537 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:46.345472 (+ 8935us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.66:52763: BlockingRecv error: recv got EOF from 127.7.111.66:52763 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 11 12:47:46 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:46 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:46.873301 23234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:46.864559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.65:53497)
0811 12:47:46.865361 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:46.865400 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:46.865530 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:46.865987 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:46.865993 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:46.866440 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:46.866626 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:46.866635 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:46.867635 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:46.867639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:46.868560 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:46.868581 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:46.869227 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:46.869231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:46.869350 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:46.869927 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:46.869944 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:46.873072 (+ 3128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":372,"threads_started":1}
W20260811 12:47:46.873605 23235 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:46.864733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:53497 (local address 127.7.111.126:38643)
0811 12:47:46.865466 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:46.865472 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:46.865492 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:47:46.865647 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:46.865652 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:46.865815 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:46.865924 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:46.866745 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:46.867516 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:46.868695 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:46.869115 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:46.869206 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:46.873285 (+ 4079us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.65:53497: BlockingRecv error: recv got EOF from 127.7.111.65:53497 (error 108)
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":464,"threads_started":1}
Aug 11 12:47:46 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:46 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:46.913273 23236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:46.904170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.67:57603)
0811 12:47:46.904699 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:46.904716 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:46.904852 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:46.905212 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:46.905217 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:46.905642 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:46.905813 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:46.905822 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:46.906855 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:46.906859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:46.907848 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:46.907860 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:46.908568 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:46.908575 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:46.908735 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:46.909339 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:46.909358 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:46.912936 (+ 3578us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":289,"threads_started":1}
W20260811 12:47:46.913341 23235 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:46.904267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:57603 (local address 127.7.111.126:38643)
0811 12:47:46.904458 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:46.904464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:46.904808 (+ 344us) server_negotiation.cc:407] Connection header received
0811 12:47:46.904984 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:46.904990 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:46.905061 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:46.905140 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:46.905964 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:46.906714 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:46.907989 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:46.908418 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:46.908527 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:46.913125 (+ 4598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.67:57603: BlockingRecv error: recv got EOF from 127.7.111.67:57603 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 12:47:47 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:47 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:47.356009 23237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:47.346488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.66:41893)
0811 12:47:47.346979 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:47.347002 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:47.347128 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:47.347430 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:47.347433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:47.347873 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:47.348075 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:47.348085 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:47.349178 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:47.349183 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:47.350262 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:47.350278 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:47.350977 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:47.350983 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:47.351146 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:47.351884 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:47.351908 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:47.355667 (+ 3759us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":280,"threads_started":1}
W20260811 12:47:47.356169 23235 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:47.346611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:41893 (local address 127.7.111.126:38643)
0811 12:47:47.346837 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:47.346843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:47.347078 (+ 235us) server_negotiation.cc:407] Connection header received
0811 12:47:47.347244 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:47.347247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:47.347312 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:47.347391 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:47.348194 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:47.349002 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:47.350391 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:47.350818 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:47.350909 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:47.355891 (+ 4982us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.66:41893: BlockingRecv error: recv got EOF from 127.7.111.66:41893 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 12:47:47 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:47 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:47.883559 23239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:47.874132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.65:38247)
0811 12:47:47.874802 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:47.874823 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:47.874977 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:47.875326 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:47.875331 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:47.875819 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:47.876000 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:47.876009 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:47.877038 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:47.877042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:47.878032 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:47.878046 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:47.878777 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:47.878783 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:47.878950 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:47.879536 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:47.879554 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:47.883210 (+ 3656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":298,"threads_started":1}
W20260811 12:47:47.883669 23241 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:47.874216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:38247 (local address 127.7.111.126:38643)
0811 12:47:47.874804 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:47.874809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:47.874896 (+ 87us) server_negotiation.cc:407] Connection header received
0811 12:47:47.875105 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:47.875109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:47.875182 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:47.875270 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:47.876153 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:47.876899 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:47.878174 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:47.878602 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:47.878719 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:47.883433 (+ 4714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.65:38247: BlockingRecv error: recv got EOF from 127.7.111.65:38247 (error 108)
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":352,"threads_started":1}
Aug 11 12:47:47 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:47 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:47.923451 23242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:47.914222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.67:54991)
0811 12:47:47.914725 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:47.914745 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:47.914866 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:47.915138 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:47.915141 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:47.915571 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:47.915762 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:47.915772 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:47.916730 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:47.916734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:47.917857 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:47.917876 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:47.918794 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:47.918801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:47.918992 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:47.919626 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:47.919645 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:47.923226 (+ 3581us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":272,"threads_started":1}
W20260811 12:47:47.923629 23241 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:47.914313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:54991 (local address 127.7.111.126:38643)
0811 12:47:47.914474 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:47.914478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:47.914811 (+ 333us) server_negotiation.cc:407] Connection header received
0811 12:47:47.914960 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:47.914964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:47.915025 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:47.915103 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:47.915883 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:47.916628 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:47.918015 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:47.918577 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:47.918708 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:47.923416 (+ 4708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.67:54991: BlockingRecv error: recv got EOF from 127.7.111.67:54991 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 12:47:48 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:48 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:48.365841 23243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:48.356898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.66:60799)
0811 12:47:48.357489 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:48.357509 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:48.357652 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:48.358004 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:48.358007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:48.358416 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:48.358599 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:48.358607 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:48.359685 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:48.359688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:48.360708 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:48.360722 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:48.361467 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:48.361470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:48.361594 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:48.362163 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:48.362179 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:48.365483 (+ 3304us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":298,"threads_started":1}
W20260811 12:47:48.365931 23241 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:48.356972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:60799 (local address 127.7.111.126:38643)
0811 12:47:48.357164 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:48.357172 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:48.357622 (+ 450us) server_negotiation.cc:407] Connection header received
0811 12:47:48.357782 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:48.357788 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:48.357876 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:48.357958 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:48.358733 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:48.359569 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:48.360836 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:48.361308 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:48.361432 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:48.365680 (+ 4248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.66:60799: BlockingRecv error: recv got EOF from 127.7.111.66:60799 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 12:47:48 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:48 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:48.893288 23246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:48.884535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.65:52061)
0811 12:47:48.885002 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:48.885018 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:48.885135 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:48.885715 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:48.885721 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:48.886210 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:48.886393 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:48.886405 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:48.887589 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:48.887593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:48.888513 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:48.888526 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:48.889185 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:48.889189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:48.889331 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:48.889914 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:48.889932 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:48.893047 (+ 3115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":246,"threads_started":1}
W20260811 12:47:48.893579 23252 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:48.884601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:52061 (local address 127.7.111.126:38643)
0811 12:47:48.885339 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:48.885344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:48.885359 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:47:48.885433 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:48.885438 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:48.885534 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:48.885634 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:48.886549 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:48.887446 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:48.888630 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:48.889056 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:48.889146 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:48.893275 (+ 4129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.65:52061: BlockingRecv error: recv got EOF from 127.7.111.65:52061 (error 108)
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":539,"threads_started":1}
Aug 11 12:47:48 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:48 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:48.933367 23254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:48.924480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.67:60029)
0811 12:47:48.925044 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:48.925060 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:48.925193 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:48.925545 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:48.925550 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:48.926012 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:48.926203 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:48.926212 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:48.927205 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:48.927210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:48.928146 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:48.928158 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:48.928893 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:48.928899 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:48.929061 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:48.929678 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:48.929696 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:48.933051 (+ 3355us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":322,"threads_started":1}
W20260811 12:47:48.933569 23252 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:48.924549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:60029 (local address 127.7.111.126:38643)
0811 12:47:48.924784 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:48.924790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:48.925145 (+ 355us) server_negotiation.cc:407] Connection header received
0811 12:47:48.925315 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:48.925321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:48.925410 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:48.925489 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:48.926349 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:48.927060 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:48.928299 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:48.928720 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:48.928825 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:48.933287 (+ 4462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.67:60029: BlockingRecv error: recv got EOF from 127.7.111.67:60029 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 12:47:49 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:49 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:49.375433 23255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:49.366756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.66:40189)
0811 12:47:49.367270 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:49.367293 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:49.367414 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:49.367718 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:49.367722 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:49.368155 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:49.368311 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:49.368320 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:49.369313 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:49.369317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:49.370315 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:49.370331 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:49.370977 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:49.370982 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:49.371116 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:49.371688 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:49.371706 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:49.375158 (+ 3452us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":264,"threads_started":1}
W20260811 12:47:49.375600 23252 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:49.366888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:40189 (local address 127.7.111.126:38643)
0811 12:47:49.367041 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:49.367045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:49.367358 (+ 313us) server_negotiation.cc:407] Connection header received
0811 12:47:49.367499 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:49.367502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:49.367596 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:49.367674 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:49.368419 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:49.369146 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:49.370444 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:49.370856 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:49.370947 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:49.375361 (+ 4414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.66:40189: BlockingRecv error: recv got EOF from 127.7.111.66:40189 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 12:47:49 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:49 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.65@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:49.903021 23257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:49.894553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.65:34647)
0811 12:47:49.895055 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:49.895072 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:49.895182 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:49.895457 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:49.895460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:49.895850 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:49.896043 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:49.896052 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:49.896998 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:49.897001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:49.897973 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:49.897986 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:49.898632 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:49.898636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:49.898780 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:49.899327 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:49.899343 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:49.902661 (+ 3318us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":244,"threads_started":1}
W20260811 12:47:49.903157 23259 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:49.894589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:34647 (local address 127.7.111.126:38643)
0811 12:47:49.895030 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:49.895034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:49.895145 (+ 111us) server_negotiation.cc:407] Connection header received
0811 12:47:49.895261 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:49.895264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:49.895340 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:49.895426 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:49.896155 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:49.896890 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:49.898100 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:49.898526 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:49.898616 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:49.902909 (+ 4293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.65:34647: BlockingRecv error: recv got EOF from 127.7.111.65:34647 (error 108)
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":240,"threads_started":1}
Aug 11 12:47:49 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:49 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.67@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:49.943207 23260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:49.934498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.67:56269)
0811 12:47:49.934985 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:49.935006 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:49.935125 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:49.935407 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:49.935410 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:49.935813 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:49.935968 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:49.935976 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:49.936889 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:49.936893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:49.937832 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:49.937845 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:49.938422 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:49.938425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:49.938547 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:49.939075 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:49.939094 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:49.942978 (+ 3884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":266,"threads_started":1}
W20260811 12:47:49.943396 23259 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:49.934560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:56269 (local address 127.7.111.126:38643)
0811 12:47:49.934858 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:49.934864 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:49.935080 (+ 216us) server_negotiation.cc:407] Connection header received
0811 12:47:49.935234 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:49.935238 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:49.935301 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:49.935374 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:49.936071 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:49.936784 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:49.937934 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:49.938317 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:49.938403 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:49.943184 (+ 4781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.67:56269: BlockingRecv error: recv got EOF from 127.7.111.67:56269 (error 108)
Metrics: {"server-negotiator.queue_time_us":149}
Aug 11 12:47:50 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:47:50 dist-test-slave-f0tn krb5kdc[21521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.111.66@KRBTEST.COM for oryx/127.7.111.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:47:50.385038 23265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:50.376721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:38643 (local address 127.7.111.66:41443)
0811 12:47:50.377179 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:50.377200 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:50.377314 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:50.377625 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:50.377628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:50.378016 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:50.378215 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:50.378225 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:50.379165 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:50.379168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:50.380087 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:50.380099 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:50.380705 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:50.380708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:50.380836 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:50.381372 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:50.381421 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:50.384777 (+ 3356us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.111.126:38643: Server oryx/127.7.111.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":228,"threads_started":1}
W20260811 12:47:50.385171 23259 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:50.376794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:41443 (local address 127.7.111.126:38643)
0811 12:47:50.376992 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:50.376999 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:50.377274 (+ 275us) server_negotiation.cc:407] Connection header received
0811 12:47:50.377427 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:50.377431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:50.377513 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:50.377591 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:50.378325 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:50.379061 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:50.380188 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:50.380593 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:50.380677 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:50.384961 (+ 4284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.111.66:41443: BlockingRecv error: recv got EOF from 127.7.111.66:41443 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:47:50.788555 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 22468
I20260811 12:47:50.800722 23271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:50.789787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57744 (local address 127.7.111.65:44201)
0811 12:47:50.790437 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:50.790445 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:50.790475 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:47:50.790541 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:50.790548 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:50.790794 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:50.790965 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:50.792369 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:50.793455 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:50.794565 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:50.795118 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:50.795237 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:50.797652 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:50.797662 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:50.797678 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:50.797737 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:50.800212 (+ 2475us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.111.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 12:47:50.800501 (+ 289us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57744: Request ticket server kudu/127.7.111.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":308,"threads_started":1}
W20260811 12:47:50.800827 23271 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57744: Request ticket server kudu/127.7.111.65@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 12:47:50.800906 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 22468: Not authorized: Client connection negotiation failed: client connection to 127.7.111.65:44201: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.111.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 12:47:50.801157 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 22468
I20260811 12:47:50.831656 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 22712
I20260811 12:47:50.844710 23273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:50.832971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57490 (local address 127.7.111.66:46859)
0811 12:47:50.833730 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:50.833740 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:50.833780 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:47:50.833860 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:50.833868 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:50.834208 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:50.834440 (+ 232us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:50.835669 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:50.837242 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:50.838515 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:50.839111 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:50.839214 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:50.841641 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:50.841650 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:50.841666 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:50.841718 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:50.844140 (+ 2422us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.111.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 12:47:50.844442 (+ 302us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57490: Request ticket server kudu/127.7.111.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":324,"threads_started":1}
W20260811 12:47:50.844827 23273 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57490: Request ticket server kudu/127.7.111.66@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 12:47:50.844835 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 22712: Not authorized: Client connection negotiation failed: client connection to 127.7.111.66:46859: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.111.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 12:47:50.845044 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 22712
I20260811 12:47:50.871909 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 22946
I20260811 12:47:50.884310 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:50.873093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52990 (local address 127.7.111.67:43097)
0811 12:47:50.873712 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:50.873720 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:50.873748 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:47:50.873812 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:50.873819 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:50.874060 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:50.874238 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:50.875996 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:50.877022 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:50.878259 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:50.878817 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:50.878920 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:50.881269 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:50.881277 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:50.881292 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:50.881336 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:50.883767 (+ 2431us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.111.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 12:47:50.884058 (+ 291us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52990: Request ticket server kudu/127.7.111.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":293,"threads_started":1}
W20260811 12:47:50.884411 23276 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52990: Request ticket server kudu/127.7.111.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 12:47:50.884459 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 22946: Not authorized: Client connection negotiation failed: client connection to 127.7.111.67:43097: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.111.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 12:47:50.884663 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 22946
I20260811 12:47:50.911675 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 22348
I20260811 12:47:50.913120 22456 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:51.031620 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 22348
2026-08-11T12:47:51Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21727 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T12:47:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:47:51Z Disabled control of system clock
I20260811 12:47:51.133509 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:44923
--webserver_interface=127.7.111.126
--webserver_port=0
--builtin_ntp_servers=127.7.111.84:33443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:44923
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 12:47:51.481372 23285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:51.481731 23285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:51.481848 23285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:51.489078 23285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:47:51.489199 23285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:51.489248 23285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:51.489332 23285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:47:51.489403 23285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:47:51.498800 23285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:33443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:44923
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:44923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:51.500221 23285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:51.502130 23285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:51.513877 23291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:51.513890 23294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:51.513866 23292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:51.514760 23285 server_base.cc:1034] running on GCE node
I20260811 12:47:51.515583 23285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:51.517016 23285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:51.518217 23285 hybrid_clock.cc:648] HybridClock initialized: now 1786452471518152 us; error 48 us; skew 500 ppm
I20260811 12:47:51.518551 23285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:47:51.524802 23285 webserver.cc:460] Webserver started at http://127.7.111.126:37517/ using document root <none> and password file <none>
I20260811 12:47:51.525651 23285 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:51.525774 23285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:51.526134 23285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:51.529229 23285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "1e48edd1ce1a4cde84aefaf7343f9457"
format_stamp: "Formatted at 2026-08-11 12:47:51 on dist-test-slave-f0tn"
server_key: "157439e6b3eedc6a9cd041937cb10a1e"
server_key_iv: "9e8e46a3633d6421b1c83c1a223ed598"
server_key_version: "encryptionkey@0"
I20260811 12:47:51.530169 23285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "1e48edd1ce1a4cde84aefaf7343f9457"
format_stamp: "Formatted at 2026-08-11 12:47:51 on dist-test-slave-f0tn"
server_key: "157439e6b3eedc6a9cd041937cb10a1e"
server_key_iv: "9e8e46a3633d6421b1c83c1a223ed598"
server_key_version: "encryptionkey@0"
I20260811 12:47:51.535605 23285 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:47:51.539113 23308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:51.540459 23285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:47:51.540629 23285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "1e48edd1ce1a4cde84aefaf7343f9457"
format_stamp: "Formatted at 2026-08-11 12:47:51 on dist-test-slave-f0tn"
server_key: "157439e6b3eedc6a9cd041937cb10a1e"
server_key_iv: "9e8e46a3633d6421b1c83c1a223ed598"
server_key_version: "encryptionkey@0"
I20260811 12:47:51.540835 23285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:51.558532 23285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:51.559497 23285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:51.559796 23285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:51.578258 23285 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:44923
I20260811 12:47:51.578286 23400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:44923 every 8 connection(s)
I20260811 12:47:51.579725 23285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:47:51.582438 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 23285
I20260811 12:47:51.582628 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:47:51.583312 7613 external_mini_cluster.cc:1428] Setting key 3f5e13cc99c4f640b6fa6bb9569b2034
I20260811 12:47:51.584018 23401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:51.599236 23401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e48edd1ce1a4cde84aefaf7343f9457: Bootstrap starting.
I20260811 12:47:51.599565 23404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:51.585501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47380 (local address 127.7.111.126:44923)
0811 12:47:51.586547 (+ 1046us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:51.586559 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:51.586606 (+ 47us) server_negotiation.cc:407] Connection header received
0811 12:47:51.589022 (+ 2416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:51.589053 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:51.589728 (+ 675us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:51.590148 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:51.590955 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:51.592376 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:51.594328 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:51.595056 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:51.595244 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:51.596684 (+ 1440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:51.596693 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:51.596712 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:47:51.597540 (+ 828us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:51.597649 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:47:51.597676 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:51.597783 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:51.598034 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 12:47:51.598426 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":361,"threads_started":1}
I20260811 12:47:51.603343 23401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e48edd1ce1a4cde84aefaf7343f9457: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:51.604547 23401 log.cc:824] T 00000000000000000000000000000000 P 1e48edd1ce1a4cde84aefaf7343f9457: Log is configured to *not* fsync() on all Append() calls
I20260811 12:47:51.608139 23401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e48edd1ce1a4cde84aefaf7343f9457: No bootstrap required, opened a new log
I20260811 12:47:51.616221 23401 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e48edd1ce1a4cde84aefaf7343f9457 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e48edd1ce1a4cde84aefaf7343f9457" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 44923 } }
I20260811 12:47:51.616679 23401 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e48edd1ce1a4cde84aefaf7343f9457 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:51.616789 23401 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e48edd1ce1a4cde84aefaf7343f9457 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e48edd1ce1a4cde84aefaf7343f9457, State: Initialized, Role: FOLLOWER
I20260811 12:47:51.617514 23401 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e48edd1ce1a4cde84aefaf7343f9457 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e48edd1ce1a4cde84aefaf7343f9457" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 44923 } }
I20260811 12:47:51.617817 23401 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e48edd1ce1a4cde84aefaf7343f9457 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:47:51.617980 23401 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e48edd1ce1a4cde84aefaf7343f9457 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:47:51.618173 23401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e48edd1ce1a4cde84aefaf7343f9457 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:51.620496 23401 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e48edd1ce1a4cde84aefaf7343f9457 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e48edd1ce1a4cde84aefaf7343f9457" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 44923 } }
I20260811 12:47:51.620998 23401 leader_election.cc:302] T 00000000000000000000000000000000 P 1e48edd1ce1a4cde84aefaf7343f9457 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e48edd1ce1a4cde84aefaf7343f9457; no voters:
I20260811 12:47:51.621635 23401 leader_election.cc:288] T 00000000000000000000000000000000 P 1e48edd1ce1a4cde84aefaf7343f9457 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:47:51.621797 23408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e48edd1ce1a4cde84aefaf7343f9457 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:47:51.622510 23408 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e48edd1ce1a4cde84aefaf7343f9457 [term 1 LEADER]: Becoming Leader. State: Replica: 1e48edd1ce1a4cde84aefaf7343f9457, State: Running, Role: LEADER
I20260811 12:47:51.623131 23408 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e48edd1ce1a4cde84aefaf7343f9457 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e48edd1ce1a4cde84aefaf7343f9457" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 44923 } }
I20260811 12:47:51.623816 23401 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e48edd1ce1a4cde84aefaf7343f9457 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:47:51.626312 23412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e48edd1ce1a4cde84aefaf7343f9457 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e48edd1ce1a4cde84aefaf7343f9457" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e48edd1ce1a4cde84aefaf7343f9457" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 44923 } } }
I20260811 12:47:51.627398 23412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e48edd1ce1a4cde84aefaf7343f9457 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:51.626305 23413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e48edd1ce1a4cde84aefaf7343f9457 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e48edd1ce1a4cde84aefaf7343f9457. Latest consensus state: current_term: 1 leader_uuid: "1e48edd1ce1a4cde84aefaf7343f9457" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e48edd1ce1a4cde84aefaf7343f9457" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 44923 } } }
I20260811 12:47:51.628947 23413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e48edd1ce1a4cde84aefaf7343f9457 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:51.630036 23429 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:47:51.636143 23429 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:47:51.645438 23429 catalog_manager.cc:1347] Generated new cluster ID: 7fa3f7c7adbf4476b7ff12f34e015a07
I20260811 12:47:51.645695 23429 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:47:51.656242 23429 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:47:51.658324 23429 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:47:51.670261 23429 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e48edd1ce1a4cde84aefaf7343f9457: Generated new TSK 0
I20260811 12:47:51.671523 23429 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:47:51.688954 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:44923
--builtin_ntp_servers=127.7.111.84:33443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 12:47:52.038168 23450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:52.038482 23450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:52.038617 23450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:52.045990 23450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:52.046111 23450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:52.046258 23450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:47:52.055441 23450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:33443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:44923
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:52.056797 23450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:52.058701 23450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:52.072319 23459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:52.072286 23462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:52.072352 23460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:52.073117 23450 server_base.cc:1034] running on GCE node
I20260811 12:47:52.073796 23450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:52.074822 23450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:52.076059 23450 hybrid_clock.cc:648] HybridClock initialized: now 1786452472076007 us; error 50 us; skew 500 ppm
I20260811 12:47:52.076439 23450 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:47:52.078864 23450 webserver.cc:460] Webserver started at http://127.7.111.65:40905/ using document root <none> and password file <none>
I20260811 12:47:52.079591 23450 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:52.079691 23450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:52.080022 23450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:52.083035 23450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "b02edda88d7c4387aa9ccc038c3a4a90"
format_stamp: "Formatted at 2026-08-11 12:47:52 on dist-test-slave-f0tn"
server_key: "331b6d0a0e1e5e146924097e77388e0f"
server_key_iv: "474128508cd281ba162f1e060ff921b2"
server_key_version: "encryptionkey@0"
I20260811 12:47:52.083833 23450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "b02edda88d7c4387aa9ccc038c3a4a90"
format_stamp: "Formatted at 2026-08-11 12:47:52 on dist-test-slave-f0tn"
server_key: "331b6d0a0e1e5e146924097e77388e0f"
server_key_iv: "474128508cd281ba162f1e060ff921b2"
server_key_version: "encryptionkey@0"
I20260811 12:47:52.089193 23450 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:47:52.092744 23472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:52.094265 23450 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:47:52.094427 23450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "b02edda88d7c4387aa9ccc038c3a4a90"
format_stamp: "Formatted at 2026-08-11 12:47:52 on dist-test-slave-f0tn"
server_key: "331b6d0a0e1e5e146924097e77388e0f"
server_key_iv: "474128508cd281ba162f1e060ff921b2"
server_key_version: "encryptionkey@0"
I20260811 12:47:52.094632 23450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:52.129346 23450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:52.130388 23450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:52.130661 23450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:52.131670 23450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:52.133158 23450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:52.133268 23450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:52.133392 23450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:52.133471 23450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 12:47:52.164592 23450 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:35265
I20260811 12:47:52.164706 23661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:35265 every 8 connection(s)
I20260811 12:47:52.166013 23450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:47:52.171855 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 23450
I20260811 12:47:52.172060 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:47:52.172880 7613 external_mini_cluster.cc:1428] Setting key 193147202434743e430e23545d12a425
I20260811 12:47:52.175915 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:44923
--builtin_ntp_servers=127.7.111.84:33443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 12:47:52.179647 23664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:52.168513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:56369 (local address 127.7.111.126:44923)
0811 12:47:52.168975 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:52.168979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:52.170286 (+ 1307us) server_negotiation.cc:407] Connection header received
0811 12:47:52.172366 (+ 2080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:52.172370 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:52.172441 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:52.172542 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:52.173934 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:52.174908 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:52.176554 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:52.177123 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:52.177244 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:52.178917 (+ 1673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:52.178923 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:52.178926 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:47:52.178944 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:52.178986 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:47:52.178999 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:52.179071 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:52.179295 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 12:47:52.179376 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":238,"threads_started":1}
I20260811 12:47:52.180959 23665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:52.169082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:44923 (local address 127.7.111.65:56369)
0811 12:47:52.170136 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:52.170174 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:52.172055 (+ 1881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:52.172813 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:52.172824 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:52.172941 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:52.173757 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:52.173778 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:52.175064 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:52.175068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:52.176395 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:52.176408 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:52.177548 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:52.177556 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:52.177837 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:52.178556 (+ 719us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:47:52.178559 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:52.178614 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:47:52.178721 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:52.179091 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:52.179101 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:52.179110 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 12:47:52.179276 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 12:47:52.179662 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":339,"threads_started":1}
I20260811 12:47:52.182456 23662 heartbeater.cc:347] Connected to a master server at 127.7.111.126:44923
I20260811 12:47:52.182776 23662 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:52.183683 23662 heartbeater.cc:511] Master 127.7.111.126:44923 requested a full tablet report, sending...
I20260811 12:47:52.185916 23338 ts_manager.cc:194] Registered new tserver with Master: b02edda88d7c4387aa9ccc038c3a4a90 (127.7.111.65:35265)
I20260811 12:47:52.187446 23338 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.111.65:56369
W20260811 12:47:52.530393 23666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:52.530723 23666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:52.530846 23666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:52.538172 23666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:52.538261 23666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:52.538371 23666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:47:52.547434 23666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:33443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:44923
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:52.548703 23666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:52.550698 23666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:52.563714 23682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:52.563591 23685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:52.563666 23681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:52.564908 23666 server_base.cc:1034] running on GCE node
I20260811 12:47:52.565573 23666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:52.566510 23666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:52.567727 23666 hybrid_clock.cc:648] HybridClock initialized: now 1786452472567673 us; error 36 us; skew 500 ppm
I20260811 12:47:52.568033 23666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:47:52.570789 23666 webserver.cc:460] Webserver started at http://127.7.111.66:45049/ using document root <none> and password file <none>
I20260811 12:47:52.571731 23666 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:52.571857 23666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:52.572253 23666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:52.576673 23666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "c2bdf8bcd89341699b213c3d6a41c31e"
format_stamp: "Formatted at 2026-08-11 12:47:52 on dist-test-slave-f0tn"
server_key: "636459de64ffd9d86950d77d839cc210"
server_key_iv: "4de96a0ae0364aa25b95b25a5ed76d66"
server_key_version: "encryptionkey@0"
I20260811 12:47:52.577786 23666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "c2bdf8bcd89341699b213c3d6a41c31e"
format_stamp: "Formatted at 2026-08-11 12:47:52 on dist-test-slave-f0tn"
server_key: "636459de64ffd9d86950d77d839cc210"
server_key_iv: "4de96a0ae0364aa25b95b25a5ed76d66"
server_key_version: "encryptionkey@0"
I20260811 12:47:52.584877 23666 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260811 12:47:52.591828 23693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:52.593755 23666 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.005s sys 0.000s
I20260811 12:47:52.593986 23666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "c2bdf8bcd89341699b213c3d6a41c31e"
format_stamp: "Formatted at 2026-08-11 12:47:52 on dist-test-slave-f0tn"
server_key: "636459de64ffd9d86950d77d839cc210"
server_key_iv: "4de96a0ae0364aa25b95b25a5ed76d66"
server_key_version: "encryptionkey@0"
I20260811 12:47:52.594295 23666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:52.616842 23666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:52.618026 23666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:52.618410 23666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:52.619673 23666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:52.621688 23666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:52.621822 23666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:52.621943 23666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:52.622033 23666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:52.652565 23666 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:45321
I20260811 12:47:52.652604 23884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:45321 every 8 connection(s)
I20260811 12:47:52.653940 23666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:47:52.664217 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 23666
I20260811 12:47:52.664361 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:47:52.664850 7613 external_mini_cluster.cc:1428] Setting key 494e73f44ed5f3f2437afd57a9b6e83a
I20260811 12:47:52.666088 23664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:52.656425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:49411 (local address 127.7.111.126:44923)
0811 12:47:52.656596 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:52.656600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:52.657929 (+ 1329us) server_negotiation.cc:407] Connection header received
0811 12:47:52.659684 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:52.659690 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:52.659776 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:52.659864 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:52.660949 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:52.661774 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:52.663315 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:52.663716 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:52.663830 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:52.665244 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:52.665249 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:52.665252 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:47:52.665270 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:52.665305 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:47:52.665317 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:52.665421 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:52.665645 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 12:47:52.665757 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:47:52.667164 23895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:52.656819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:44923 (local address 127.7.111.66:49411)
0811 12:47:52.657771 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:52.657809 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:52.659404 (+ 1595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:52.660108 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:52.660116 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:52.660199 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:52.660781 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:52.660799 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:52.661924 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:52.661928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:52.663165 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:52.663177 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:52.663953 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:52.663961 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:52.664174 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:52.664884 (+ 710us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:47:52.664887 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:52.664943 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:47:52.665038 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:52.665432 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:52.665444 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:52.665453 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 12:47:52.665620 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 12:47:52.666037 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":277,"threads_started":1}
I20260811 12:47:52.667850 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:44923
--builtin_ntp_servers=127.7.111.84:33443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 12:47:52.668578 23887 heartbeater.cc:347] Connected to a master server at 127.7.111.126:44923
I20260811 12:47:52.668915 23887 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:52.669898 23887 heartbeater.cc:511] Master 127.7.111.126:44923 requested a full tablet report, sending...
I20260811 12:47:52.671494 23338 ts_manager.cc:194] Registered new tserver with Master: c2bdf8bcd89341699b213c3d6a41c31e (127.7.111.66:45321)
I20260811 12:47:52.672456 23338 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.111.66:49411
W20260811 12:47:53.011488 23896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:53.011785 23896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:53.011873 23896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:53.019150 23896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:53.019248 23896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:53.019363 23896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:47:53.028980 23896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:33443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:44923
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:53.030318 23896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:53.032200 23896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:53.045653 23901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:53.045660 23902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:53.046437 23904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:53.046766 23896 server_base.cc:1034] running on GCE node
I20260811 12:47:53.047386 23896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:53.048362 23896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:53.049575 23896 hybrid_clock.cc:648] HybridClock initialized: now 1786452473049509 us; error 48 us; skew 500 ppm
I20260811 12:47:53.049948 23896 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:47:53.052299 23896 webserver.cc:460] Webserver started at http://127.7.111.67:42639/ using document root <none> and password file <none>
I20260811 12:47:53.053026 23896 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:53.053134 23896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:53.053507 23896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:53.056449 23896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "5cbbd804b4044b6a8d0632d3705df115"
format_stamp: "Formatted at 2026-08-11 12:47:53 on dist-test-slave-f0tn"
server_key: "d91469107cdd40124e638edb2bf7eaaf"
server_key_iv: "7318fc9451d3ef184ade691d1ec9659b"
server_key_version: "encryptionkey@0"
I20260811 12:47:53.057238 23896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "5cbbd804b4044b6a8d0632d3705df115"
format_stamp: "Formatted at 2026-08-11 12:47:53 on dist-test-slave-f0tn"
server_key: "d91469107cdd40124e638edb2bf7eaaf"
server_key_iv: "7318fc9451d3ef184ade691d1ec9659b"
server_key_version: "encryptionkey@0"
I20260811 12:47:53.062305 23896 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 12:47:53.065641 23916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:53.066995 23896 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:47:53.067143 23896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "5cbbd804b4044b6a8d0632d3705df115"
format_stamp: "Formatted at 2026-08-11 12:47:53 on dist-test-slave-f0tn"
server_key: "d91469107cdd40124e638edb2bf7eaaf"
server_key_iv: "7318fc9451d3ef184ade691d1ec9659b"
server_key_version: "encryptionkey@0"
I20260811 12:47:53.067317 23896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:53.080598 23896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:53.081434 23896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:53.081704 23896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:53.082605 23896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:53.084000 23896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:53.084089 23896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:53.084182 23896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:53.084237 23896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:53.115000 23896 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:41845
I20260811 12:47:53.115053 24123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:41845 every 8 connection(s)
I20260811 12:47:53.116349 23896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:47:53.117512 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 23896
I20260811 12:47:53.117689 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:47:53.118274 7613 external_mini_cluster.cc:1428] Setting key f33e433a56f76a386449a4f101ddc085
I20260811 12:47:53.129220 23664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:53.119422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:58447 (local address 127.7.111.126:44923)
0811 12:47:53.119605 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:53.119614 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:53.121279 (+ 1665us) server_negotiation.cc:407] Connection header received
0811 12:47:53.123218 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:53.123223 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:53.123292 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:53.123391 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:53.124428 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:53.125191 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:53.126786 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:53.127215 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:53.127321 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:53.128582 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:53.128586 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:53.128590 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:47:53.128606 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:53.128636 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:47:53.128648 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:53.128714 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:53.128888 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 12:47:53.128964 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 12:47:53.130239 24126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:53.119999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:44923 (local address 127.7.111.67:58447)
0811 12:47:53.121115 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:53.121166 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:53.122967 (+ 1801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:53.123582 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:53.123590 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:53.123691 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:53.124284 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:53.124303 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:53.125312 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:53.125316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:53.126634 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:53.126648 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:53.127396 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:53.127401 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:53.127608 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:53.128281 (+ 673us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:47:53.128284 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:53.128325 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:47:53.128403 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:53.128720 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:53.128726 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:53.128732 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 12:47:53.128872 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 12:47:53.129157 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":295,"threads_started":1}
I20260811 12:47:53.131659 24124 heartbeater.cc:347] Connected to a master server at 127.7.111.126:44923
I20260811 12:47:53.131937 24124 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:53.132810 24124 heartbeater.cc:511] Master 127.7.111.126:44923 requested a full tablet report, sending...
I20260811 12:47:53.134400 23338 ts_manager.cc:194] Registered new tserver with Master: 5cbbd804b4044b6a8d0632d3705df115 (127.7.111.67:41845)
I20260811 12:47:53.135196 23338 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.111.67:58447
I20260811 12:47:53.146827 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 12:47:53.153512 23664 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:53.151752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47384 (local address 127.7.111.126:44923)
0811 12:47:53.151969 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:53.151976 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:53.152403 (+ 427us) server_negotiation.cc:407] Connection header received
0811 12:47:53.152574 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:53.152580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:53.152665 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:53.152752 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:53.153223 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47384: BlockingRecv error: recv got EOF from 127.0.0.1:47384 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:47:53.154538 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 23450
I20260811 12:47:53.162509 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:53.155447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53412 (local address 127.7.111.65:35265)
0811 12:47:53.156066 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:53.156073 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:53.156105 (+ 32us) server_negotiation.cc:407] Connection header received
0811 12:47:53.156178 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:53.156186 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:53.156460 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:53.156637 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:53.156971 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:53.158055 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:53.159390 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:53.159956 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:53.160088 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:53.161043 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:53.161051 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:53.161087 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:47:53.161782 (+ 695us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:53.161832 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:47:53.161853 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:53.161937 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:53.162083 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 12:47:53.162200 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":339,"threads_started":1}
I20260811 12:47:53.163801 23657 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:53.245249 23662 heartbeater.cc:503] Master 127.7.111.126:44923 was elected leader, sending a full tablet report...
I20260811 12:47:53.246659 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 23450
W20260811 12:47:53.273366 23314 connection.cc:535] server connection from 127.7.111.65:56369 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 12:47:53.273718 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 23666
I20260811 12:47:53.281924 24146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:53.274607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52876 (local address 127.7.111.66:45321)
0811 12:47:53.275204 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:53.275212 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:53.275242 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:47:53.275329 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:53.275336 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:53.275557 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:53.275708 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:53.276151 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:53.277146 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:53.278552 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:53.279134 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:53.279240 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:53.280306 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:53.280315 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:53.280331 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:47:53.280956 (+ 625us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:53.281014 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:47:53.281034 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:53.281138 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:53.281330 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 12:47:53.281532 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":290,"threads_started":1}
I20260811 12:47:53.283311 23878 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:53.366024 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 23666
I20260811 12:47:53.388996 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 23896
I20260811 12:47:53.397267 24148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:53.390205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56064 (local address 127.7.111.67:41845)
0811 12:47:53.390872 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:53.390879 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:53.390914 (+ 35us) server_negotiation.cc:407] Connection header received
0811 12:47:53.390981 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:53.390988 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:53.391229 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:53.391370 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:53.391774 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:53.392848 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:53.394226 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:53.394807 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:53.394906 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:53.395843 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:53.395852 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:53.395867 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:47:53.396483 (+ 616us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:53.396531 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:47:53.396556 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:53.396646 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:53.396741 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 12:47:53.396871 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":332,"threads_started":1}
I20260811 12:47:53.398546 24119 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:53.480618 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 23896
I20260811 12:47:53.508551 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 23285
I20260811 12:47:53.509853 23390 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:53.613312 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 23285
2026-08-11T12:47:53Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2570 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T12:47:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:47:53Z Disabled control of system clock
I20260811 12:47:53.703900 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:40531
--webserver_interface=127.7.111.126
--webserver_port=0
--builtin_ntp_servers=127.7.111.84:46013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:40531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 12:47:54.046238 24158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:54.046562 24158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:54.046649 24158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:54.054019 24158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:47:54.054143 24158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:54.054186 24158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:54.054245 24158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:47:54.054289 24158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:47:54.063812 24158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:46013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:40531
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:40531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:54.065167 24158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:54.067004 24158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:54.078845 24172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:54.078884 24173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:54.078915 24175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:54.079699 24158 server_base.cc:1034] running on GCE node
I20260811 12:47:54.080502 24158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:54.082031 24158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:54.083242 24158 hybrid_clock.cc:648] HybridClock initialized: now 1786452474083194 us; error 36 us; skew 500 ppm
I20260811 12:47:54.083609 24158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:47:54.085958 24158 webserver.cc:460] Webserver started at http://127.7.111.126:40351/ using document root <none> and password file <none>
I20260811 12:47:54.086707 24158 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:54.086817 24158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:54.087152 24158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:54.090201 24158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "e540a937d32140e0aafa270093a1b237"
format_stamp: "Formatted at 2026-08-11 12:47:54 on dist-test-slave-f0tn"
server_key: "84695525113ab744380df2291469c67c"
server_key_iv: "9c6b73bf0b03117d98635c4429d1b8f4"
server_key_version: "encryptionkey@0"
I20260811 12:47:54.091024 24158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "e540a937d32140e0aafa270093a1b237"
format_stamp: "Formatted at 2026-08-11 12:47:54 on dist-test-slave-f0tn"
server_key: "84695525113ab744380df2291469c67c"
server_key_iv: "9c6b73bf0b03117d98635c4429d1b8f4"
server_key_version: "encryptionkey@0"
I20260811 12:47:54.096248 24158 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260811 12:47:54.099752 24183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:54.101074 24158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:47:54.101229 24158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "e540a937d32140e0aafa270093a1b237"
format_stamp: "Formatted at 2026-08-11 12:47:54 on dist-test-slave-f0tn"
server_key: "84695525113ab744380df2291469c67c"
server_key_iv: "9c6b73bf0b03117d98635c4429d1b8f4"
server_key_version: "encryptionkey@0"
I20260811 12:47:54.101485 24158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:54.107177 24158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:54.107995 24158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:54.108276 24158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:54.126394 24158 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:40531
I20260811 12:47:54.126442 24276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:40531 every 8 connection(s)
I20260811 12:47:54.127810 24158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:47:54.131726 24278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:54.132478 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 24158
I20260811 12:47:54.132705 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:47:54.133651 7613 external_mini_cluster.cc:1428] Setting key ae437f0f3b109d6e1227d8033e43ec56
I20260811 12:47:54.142124 24289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:54.135832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43312 (local address 127.7.111.126:40531)
0811 12:47:54.136897 (+ 1065us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:54.136909 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:54.136955 (+ 46us) server_negotiation.cc:407] Connection header received
0811 12:47:54.138579 (+ 1624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:54.138603 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:54.139048 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:54.139393 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:54.140488 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:54.140494 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:54.140507 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:47:54.140521 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:54.140587 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:47:54.140603 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:54.140783 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:54.140877 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 12:47:54.141133 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":378,"threads_started":1}
I20260811 12:47:54.145437 24278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e540a937d32140e0aafa270093a1b237: Bootstrap starting.
I20260811 12:47:54.148469 24278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e540a937d32140e0aafa270093a1b237: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:54.149617 24278 log.cc:824] T 00000000000000000000000000000000 P e540a937d32140e0aafa270093a1b237: Log is configured to *not* fsync() on all Append() calls
I20260811 12:47:54.152581 24278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e540a937d32140e0aafa270093a1b237: No bootstrap required, opened a new log
I20260811 12:47:54.159937 24278 raft_consensus.cc:348] T 00000000000000000000000000000000 P e540a937d32140e0aafa270093a1b237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e540a937d32140e0aafa270093a1b237" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40531 } }
I20260811 12:47:54.160298 24278 raft_consensus.cc:374] T 00000000000000000000000000000000 P e540a937d32140e0aafa270093a1b237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:54.160396 24278 raft_consensus.cc:729] T 00000000000000000000000000000000 P e540a937d32140e0aafa270093a1b237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e540a937d32140e0aafa270093a1b237, State: Initialized, Role: FOLLOWER
I20260811 12:47:54.160975 24278 consensus_queue.cc:260] T 00000000000000000000000000000000 P e540a937d32140e0aafa270093a1b237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e540a937d32140e0aafa270093a1b237" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40531 } }
I20260811 12:47:54.161188 24278 raft_consensus.cc:388] T 00000000000000000000000000000000 P e540a937d32140e0aafa270093a1b237 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:47:54.161326 24278 raft_consensus.cc:482] T 00000000000000000000000000000000 P e540a937d32140e0aafa270093a1b237 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:47:54.161495 24278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e540a937d32140e0aafa270093a1b237 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:54.163664 24278 raft_consensus.cc:504] T 00000000000000000000000000000000 P e540a937d32140e0aafa270093a1b237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e540a937d32140e0aafa270093a1b237" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40531 } }
I20260811 12:47:54.164191 24278 leader_election.cc:302] T 00000000000000000000000000000000 P e540a937d32140e0aafa270093a1b237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e540a937d32140e0aafa270093a1b237; no voters:
I20260811 12:47:54.164762 24278 leader_election.cc:288] T 00000000000000000000000000000000 P e540a937d32140e0aafa270093a1b237 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:47:54.164897 24292 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e540a937d32140e0aafa270093a1b237 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:47:54.165616 24292 raft_consensus.cc:686] T 00000000000000000000000000000000 P e540a937d32140e0aafa270093a1b237 [term 1 LEADER]: Becoming Leader. State: Replica: e540a937d32140e0aafa270093a1b237, State: Running, Role: LEADER
I20260811 12:47:54.166159 24292 consensus_queue.cc:237] T 00000000000000000000000000000000 P e540a937d32140e0aafa270093a1b237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e540a937d32140e0aafa270093a1b237" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40531 } }
I20260811 12:47:54.166795 24278 sys_catalog.cc:564] T 00000000000000000000000000000000 P e540a937d32140e0aafa270093a1b237 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:47:54.169220 24298 sys_catalog.cc:455] T 00000000000000000000000000000000 P e540a937d32140e0aafa270093a1b237 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e540a937d32140e0aafa270093a1b237. Latest consensus state: current_term: 1 leader_uuid: "e540a937d32140e0aafa270093a1b237" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e540a937d32140e0aafa270093a1b237" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40531 } } }
I20260811 12:47:54.169656 24298 sys_catalog.cc:458] T 00000000000000000000000000000000 P e540a937d32140e0aafa270093a1b237 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:54.169552 24295 sys_catalog.cc:455] T 00000000000000000000000000000000 P e540a937d32140e0aafa270093a1b237 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e540a937d32140e0aafa270093a1b237" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e540a937d32140e0aafa270093a1b237" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40531 } } }
I20260811 12:47:54.169879 24295 sys_catalog.cc:458] T 00000000000000000000000000000000 P e540a937d32140e0aafa270093a1b237 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:54.171777 24306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:47:54.176282 24306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:47:54.184499 24306 catalog_manager.cc:1347] Generated new cluster ID: 5af5e1554d9b4886a48e2b9c6e624e05
I20260811 12:47:54.184628 24306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:47:54.195907 24306 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:47:54.196437 24306 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:47:54.205726 24306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e540a937d32140e0aafa270093a1b237: Generated new TSK 0
I20260811 12:47:54.206550 24306 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:47:54.212208 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40531
--builtin_ntp_servers=127.7.111.84:46013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 12:47:54.556717 24323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:54.557034 24323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:54.557157 24323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:54.564610 24323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:54.564730 24323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:54.564878 24323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:47:54.574520 24323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:46013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:54.575929 24323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:54.577844 24323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:54.592706 24332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:54.592744 24333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:54.592767 24335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:54.594035 24323 server_base.cc:1034] running on GCE node
I20260811 12:47:54.594831 24323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:54.599077 24323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:54.600447 24323 hybrid_clock.cc:648] HybridClock initialized: now 1786452474600321 us; error 126 us; skew 500 ppm
I20260811 12:47:54.600821 24323 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:47:54.603629 24323 webserver.cc:460] Webserver started at http://127.7.111.65:44801/ using document root <none> and password file <none>
I20260811 12:47:54.604418 24323 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:54.604554 24323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:54.604913 24323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:54.608134 24323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "4346ee421689412f86f9b20edeceeef2"
format_stamp: "Formatted at 2026-08-11 12:47:54 on dist-test-slave-f0tn"
server_key: "194275cadd2dda32eea4858e17c1dc32"
server_key_iv: "b0f7c88aec6d920a7ba0823db7bed7f0"
server_key_version: "encryptionkey@0"
I20260811 12:47:54.608990 24323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "4346ee421689412f86f9b20edeceeef2"
format_stamp: "Formatted at 2026-08-11 12:47:54 on dist-test-slave-f0tn"
server_key: "194275cadd2dda32eea4858e17c1dc32"
server_key_iv: "b0f7c88aec6d920a7ba0823db7bed7f0"
server_key_version: "encryptionkey@0"
I20260811 12:47:54.614775 24323 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 12:47:54.618319 24342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:54.619630 24323 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:47:54.619808 24323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "4346ee421689412f86f9b20edeceeef2"
format_stamp: "Formatted at 2026-08-11 12:47:54 on dist-test-slave-f0tn"
server_key: "194275cadd2dda32eea4858e17c1dc32"
server_key_iv: "b0f7c88aec6d920a7ba0823db7bed7f0"
server_key_version: "encryptionkey@0"
I20260811 12:47:54.620015 24323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:54.625936 24323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:54.626793 24323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:54.627045 24323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:54.627988 24323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:54.629503 24323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:54.629594 24323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:54.629680 24323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:54.629736 24323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:54.657841 24323 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:33319
I20260811 12:47:54.657887 24536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:33319 every 8 connection(s)
I20260811 12:47:54.659132 24323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:47:54.663355 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 24323
I20260811 12:47:54.663587 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:47:54.664431 7613 external_mini_cluster.cc:1428] Setting key 33685fe0f707f018c48eafa43debf618
I20260811 12:47:54.667608 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40531
--builtin_ntp_servers=127.7.111.84:46013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 12:47:54.668321 24552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:54.661802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:58231 (local address 127.7.111.126:40531)
0811 12:47:54.662266 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:54.662270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:54.663721 (+ 1451us) server_negotiation.cc:407] Connection header received
0811 12:47:54.665934 (+ 2213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:54.665939 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:54.665997 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:54.666112 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:54.667554 (+ 1442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:54.667561 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:54.667566 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:47:54.667588 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:54.667633 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:47:54.667645 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:54.667764 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:54.667974 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 12:47:54.668027 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":232,"threads_started":1}
I20260811 12:47:54.669988 24554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:54.662416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40531 (local address 127.7.111.65:58231)
0811 12:47:54.663511 (+ 1095us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:54.663560 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:54.665668 (+ 2108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:54.666338 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:54.666349 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:54.666460 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:54.667167 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:47:54.667171 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:54.667233 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:47:54.667370 (+ 137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:54.667754 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:54.667767 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:54.667776 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 12:47:54.668472 (+ 696us) client_negotiation.cc:240] Negotiation successful
0811 12:47:54.668764 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":261,"threads_started":1}
I20260811 12:47:54.671691 24540 heartbeater.cc:347] Connected to a master server at 127.7.111.126:40531
I20260811 12:47:54.672003 24540 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:54.672611 24540 heartbeater.cc:511] Master 127.7.111.126:40531 requested a full tablet report, sending...
I20260811 12:47:54.674953 24213 ts_manager.cc:194] Registered new tserver with Master: 4346ee421689412f86f9b20edeceeef2 (127.7.111.65:33319)
W20260811 12:47:55.016407 24555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:55.016747 24555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:55.016912 24555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:55.024598 24555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:55.024740 24555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:55.024916 24555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:47:55.034679 24555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:46013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:55.036110 24555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:55.038148 24555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:55.051712 24574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:55.051775 24569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:55.051750 24567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:55.052371 24555 server_base.cc:1034] running on GCE node
I20260811 12:47:55.052970 24555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:55.053933 24555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:55.055150 24555 hybrid_clock.cc:648] HybridClock initialized: now 1786452475055079 us; error 56 us; skew 500 ppm
I20260811 12:47:55.055473 24555 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:47:55.057797 24555 webserver.cc:460] Webserver started at http://127.7.111.66:43911/ using document root <none> and password file <none>
I20260811 12:47:55.058525 24555 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:55.058627 24555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:55.058912 24555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:55.061885 24555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "6dd766b825be4dd8b36ef03025001afd"
format_stamp: "Formatted at 2026-08-11 12:47:55 on dist-test-slave-f0tn"
server_key: "ea6c12c0228862848a7f6579d93ec80b"
server_key_iv: "a2509e247bd551d8f3d4f685386b7bb2"
server_key_version: "encryptionkey@0"
I20260811 12:47:55.062664 24555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "6dd766b825be4dd8b36ef03025001afd"
format_stamp: "Formatted at 2026-08-11 12:47:55 on dist-test-slave-f0tn"
server_key: "ea6c12c0228862848a7f6579d93ec80b"
server_key_iv: "a2509e247bd551d8f3d4f685386b7bb2"
server_key_version: "encryptionkey@0"
I20260811 12:47:55.068115 24555 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 12:47:55.071998 24581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:55.073500 24555 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:47:55.073648 24555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "6dd766b825be4dd8b36ef03025001afd"
format_stamp: "Formatted at 2026-08-11 12:47:55 on dist-test-slave-f0tn"
server_key: "ea6c12c0228862848a7f6579d93ec80b"
server_key_iv: "a2509e247bd551d8f3d4f685386b7bb2"
server_key_version: "encryptionkey@0"
I20260811 12:47:55.073837 24555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:55.079427 24555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:55.080175 24555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:55.080410 24555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:55.081282 24555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:55.082712 24555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:55.082796 24555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:55.082883 24555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:55.082937 24555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:55.113415 24555 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:37213
I20260811 12:47:55.113497 24783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:37213 every 8 connection(s)
I20260811 12:47:55.114763 24555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:47:55.116894 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 24555
I20260811 12:47:55.117074 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:47:55.117658 7613 external_mini_cluster.cc:1428] Setting key c04638ea08a248aea0554f53f314e221
I20260811 12:47:55.120801 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40531
--builtin_ntp_servers=127.7.111.84:46013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 12:47:55.123744 24552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:55.117314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:36301 (local address 127.7.111.126:40531)
0811 12:47:55.117549 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:55.117555 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:55.119152 (+ 1597us) server_negotiation.cc:407] Connection header received
0811 12:47:55.121450 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:55.121454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:55.121521 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:55.121600 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:55.123068 (+ 1468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:55.123073 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:55.123076 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:47:55.123091 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:55.123122 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:47:55.123133 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:55.123233 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:55.123453 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 12:47:55.123508 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 12:47:55.124953 24788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:55.117968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40531 (local address 127.7.111.66:36301)
0811 12:47:55.119002 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:55.119039 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:55.121123 (+ 2084us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:55.121892 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:55.121904 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:55.122029 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:55.122726 (+ 697us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:47:55.122730 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:55.122789 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:47:55.122903 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:55.123227 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:55.123236 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:55.123245 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 12:47:55.123531 (+ 286us) client_negotiation.cc:240] Negotiation successful
0811 12:47:55.123803 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":320,"threads_started":1}
I20260811 12:47:55.126370 24785 heartbeater.cc:347] Connected to a master server at 127.7.111.126:40531
I20260811 12:47:55.126662 24785 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:55.127338 24785 heartbeater.cc:511] Master 127.7.111.126:40531 requested a full tablet report, sending...
I20260811 12:47:55.128723 24213 ts_manager.cc:194] Registered new tserver with Master: 6dd766b825be4dd8b36ef03025001afd (127.7.111.66:37213)
W20260811 12:47:55.460376 24792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:55.460668 24792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:55.460814 24792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:55.468060 24792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:55.468182 24792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:55.468338 24792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:47:55.477636 24792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:46013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:55.478983 24792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:55.480778 24792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:55.493952 24806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:55.494086 24803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:55.493921 24801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:55.494468 24792 server_base.cc:1034] running on GCE node
I20260811 12:47:55.495158 24792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:55.496063 24792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:55.497258 24792 hybrid_clock.cc:648] HybridClock initialized: now 1786452475497199 us; error 46 us; skew 500 ppm
I20260811 12:47:55.497579 24792 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:47:55.499730 24792 webserver.cc:460] Webserver started at http://127.7.111.67:46225/ using document root <none> and password file <none>
I20260811 12:47:55.500370 24792 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:55.500451 24792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:55.500743 24792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:55.503626 24792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "a3fc385f759e44728b219398292ee826"
format_stamp: "Formatted at 2026-08-11 12:47:55 on dist-test-slave-f0tn"
server_key: "efd1f7cabcccbfe86d9e567734f2ab69"
server_key_iv: "a9bea31d658006847feb3074aece10ba"
server_key_version: "encryptionkey@0"
I20260811 12:47:55.504412 24792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "a3fc385f759e44728b219398292ee826"
format_stamp: "Formatted at 2026-08-11 12:47:55 on dist-test-slave-f0tn"
server_key: "efd1f7cabcccbfe86d9e567734f2ab69"
server_key_iv: "a9bea31d658006847feb3074aece10ba"
server_key_version: "encryptionkey@0"
I20260811 12:47:55.509253 24792 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 12:47:55.512593 24816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:55.513876 24792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:47:55.514002 24792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "a3fc385f759e44728b219398292ee826"
format_stamp: "Formatted at 2026-08-11 12:47:55 on dist-test-slave-f0tn"
server_key: "efd1f7cabcccbfe86d9e567734f2ab69"
server_key_iv: "a9bea31d658006847feb3074aece10ba"
server_key_version: "encryptionkey@0"
I20260811 12:47:55.514166 24792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:55.519258 24792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:55.519953 24792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:55.520156 24792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:55.520975 24792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:55.522397 24792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:55.522483 24792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:55.522569 24792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:55.522622 24792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:55.552599 24792 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:36009
I20260811 12:47:55.552655 24996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:36009 every 8 connection(s)
I20260811 12:47:55.553925 24792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:47:55.561565 24552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:55.556313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:51855 (local address 127.7.111.126:40531)
0811 12:47:55.556542 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:55.556548 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:55.558049 (+ 1501us) server_negotiation.cc:407] Connection header received
0811 12:47:55.559681 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:55.559685 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:55.559740 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:55.559823 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:55.560973 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:55.560977 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:55.560980 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:47:55.560997 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:55.561030 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:47:55.561041 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:55.561142 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:55.561268 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 12:47:55.561318 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 12:47:55.562378 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 24792
I20260811 12:47:55.562585 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:47:55.562712 24999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:55.556845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40531 (local address 127.7.111.67:51855)
0811 12:47:55.557896 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:55.557940 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:55.559447 (+ 1507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:55.560007 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:55.560015 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:55.560104 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:55.560696 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:47:55.560699 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:55.560737 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:47:55.560815 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:55.561115 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:55.561120 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:55.561125 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 12:47:55.561325 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 12:47:55.561554 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":286,"threads_started":1}
I20260811 12:47:55.563136 7613 external_mini_cluster.cc:1428] Setting key c5fbdde096e695c247b47c5d1ed88143
I20260811 12:47:55.564121 24997 heartbeater.cc:347] Connected to a master server at 127.7.111.126:40531
I20260811 12:47:55.564425 24997 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:55.565083 24997 heartbeater.cc:511] Master 127.7.111.126:40531 requested a full tablet report, sending...
I20260811 12:47:55.566718 24213 ts_manager.cc:194] Registered new tserver with Master: a3fc385f759e44728b219398292ee826 (127.7.111.67:36009)
I20260811 12:47:55.578392 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 12:47:55.584242 24552 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:47:55.582818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54202 (local address 127.7.111.126:40531)
0811 12:47:55.582988 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:55.582992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:55.583470 (+ 478us) server_negotiation.cc:407] Connection header received
0811 12:47:55.583614 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:55.583618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:55.583667 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:55.583738 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:55.583987 (+ 249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54202: BlockingRecv error: recv got EOF from 127.0.0.1:54202 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:47:55.585136 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 24323
I20260811 12:47:55.588657 25009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:55.586097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48356 (local address 127.7.111.65:33319)
0811 12:47:55.586586 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:55.586593 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:55.586618 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:47:55.586674 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:55.586680 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:55.586890 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:55.587028 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:55.587979 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:55.587985 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:55.588000 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:47:55.588019 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:55.588059 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:47:55.588072 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:55.588193 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:55.588305 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 12:47:55.588405 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":238,"threads_started":1}
I20260811 12:47:55.589840 24531 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:55.670846 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 24323
I20260811 12:47:55.693625 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 24555
I20260811 12:47:55.697283 25012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:55.694594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43834 (local address 127.7.111.66:37213)
0811 12:47:55.695196 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:55.695202 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:55.695231 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:47:55.695297 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:55.695311 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:55.695539 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:55.695666 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:55.696543 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:55.696550 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:55.696562 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:47:55.696580 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:55.696622 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:47:55.696636 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:55.696754 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:55.696828 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 12:47:55.696946 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":298,"threads_started":1}
I20260811 12:47:55.698577 24777 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:55.779533 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 24555
I20260811 12:47:55.807206 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 24792
I20260811 12:47:55.810860 25015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:55.808116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57500 (local address 127.7.111.67:36009)
0811 12:47:55.808764 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:55.808771 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:55.808804 (+ 33us) server_negotiation.cc:407] Connection header received
0811 12:47:55.808871 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:55.808878 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:55.809095 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:55.809237 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:55.810176 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:55.810184 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:55.810195 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:47:55.810207 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:55.810248 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:47:55.810263 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:55.810400 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:55.810494 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 12:47:55.810588 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":317,"threads_started":1}
I20260811 12:47:55.812110 24990 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:55.892423 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 24792
I20260811 12:47:55.916908 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 24158
I20260811 12:47:55.918149 24268 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:47:56.021873 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 24158
2026-08-11T12:47:56Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2405 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:47:56 dist-test-slave-f0tn krb5kdc[25027](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:47:56 dist-test-slave-f0tn krb5kdc[25027](info): set up 2 sockets
Aug 11 12:47:56 dist-test-slave-f0tn krb5kdc[25027](info): commencing operation
krb5kdc: starting...
W20260811 12:47:57.322062 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.006s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:47:57 dist-test-slave-f0tn krb5kdc[25027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:47:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:47:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:47:57.618108 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:36359
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:37915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:36359
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:47:57.951927 25055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:57.952260 25055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:57.952404 25055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:57.959578 25055 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:47:57.959687 25055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:57.959779 25055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:57.959856 25055 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:47:57.959915 25055 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:47:57.969645 25055 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:37915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:36359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:36359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:57.970959 25055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:57.972283 25055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:57.983946 25062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:57.983996 25065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:57.984026 25067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:57.985013 25055 server_base.cc:1034] running on GCE node
I20260811 12:47:57.985855 25055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:57.987229 25055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:57.988448 25055 hybrid_clock.cc:648] HybridClock initialized: now 1786452477988385 us; error 41 us; skew 500 ppm
I20260811 12:47:57.988773 25055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:57 dist-test-slave-f0tn krb5kdc[25027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452477, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:57.995239 25055 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:47:57.996740 25055 webserver.cc:460] Webserver started at http://127.7.111.126:39937/ using document root <none> and password file <none>
I20260811 12:47:57.997422 25055 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:57.997541 25055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:57.997844 25055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:58.000818 25055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "453459e6ea1c47128c7cac11d14303d0"
format_stamp: "Formatted at 2026-08-11 12:47:57 on dist-test-slave-f0tn"
server_key: "9e36632298f043fbedf614aa48293b16"
server_key_iv: "874505a6df42477c4aa430ed0bca983c"
server_key_version: "encryptionkey@0"
I20260811 12:47:58.001608 25055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "453459e6ea1c47128c7cac11d14303d0"
format_stamp: "Formatted at 2026-08-11 12:47:57 on dist-test-slave-f0tn"
server_key: "9e36632298f043fbedf614aa48293b16"
server_key_iv: "874505a6df42477c4aa430ed0bca983c"
server_key_version: "encryptionkey@0"
I20260811 12:47:58.006728 25055 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:47:58.010074 25080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:58.011356 25055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:47:58.011526 25055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "453459e6ea1c47128c7cac11d14303d0"
format_stamp: "Formatted at 2026-08-11 12:47:57 on dist-test-slave-f0tn"
server_key: "9e36632298f043fbedf614aa48293b16"
server_key_iv: "874505a6df42477c4aa430ed0bca983c"
server_key_version: "encryptionkey@0"
I20260811 12:47:58.011718 25055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:58.032087 25055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:58.036121 25055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:58.036428 25055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:58.054261 25055 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:36359
I20260811 12:47:58.054297 25163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:36359 every 8 connection(s)
I20260811 12:47:58.055687 25055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:47:58.056128 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 25055
I20260811 12:47:58.056342 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:47:58.057112 7613 external_mini_cluster.cc:1428] Setting key b41c4908b2da69d1c7dc3e806203113c
I20260811 12:47:58.060113 25165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 12:47:58 dist-test-slave-f0tn krb5kdc[25027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:47:58.072760 25165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0: Bootstrap starting.
I20260811 12:47:58.076851 25165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:58.078064 25165 log.cc:824] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0: Log is configured to *not* fsync() on all Append() calls
I20260811 12:47:58.081333 25169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:58.059209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51478 (local address 127.7.111.126:36359)
0811 12:47:58.060003 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:58.060018 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:58.060073 (+ 55us) server_negotiation.cc:407] Connection header received
0811 12:47:58.062396 (+ 2323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:58.062425 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:58.062906 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:58.063341 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:58.064645 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:58.066825 (+ 2180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:58.068088 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:58.069052 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:58.069230 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:58.073091 (+ 3861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:58.073102 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:58.073120 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:58.073165 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:58.076866 (+ 3701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:58.077536 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:58.077544 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:58.077552 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:58.077773 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:58.078112 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:58.078116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:58.078120 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:58.078887 (+ 767us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:58.079116 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:58.079596 (+ 480us) server_negotiation.cc:299] Negotiation successful
0811 12:47:58.080031 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":321,"threads_started":1}
I20260811 12:47:58.081527 25165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0: No bootstrap required, opened a new log
I20260811 12:47:58.089432 25165 raft_consensus.cc:348] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "453459e6ea1c47128c7cac11d14303d0" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36359 } }
I20260811 12:47:58.089823 25165 raft_consensus.cc:374] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:58.089922 25165 raft_consensus.cc:729] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 453459e6ea1c47128c7cac11d14303d0, State: Initialized, Role: FOLLOWER
I20260811 12:47:58.090574 25165 consensus_queue.cc:260] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "453459e6ea1c47128c7cac11d14303d0" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36359 } }
I20260811 12:47:58.090826 25165 raft_consensus.cc:388] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:47:58.090956 25165 raft_consensus.cc:482] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:47:58.091112 25165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:47:58.093276 25165 raft_consensus.cc:504] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "453459e6ea1c47128c7cac11d14303d0" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36359 } }
I20260811 12:47:58.093798 25165 leader_election.cc:302] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 453459e6ea1c47128c7cac11d14303d0; no voters:
I20260811 12:47:58.094413 25165 leader_election.cc:288] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:47:58.094558 25172 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:47:58.095242 25172 raft_consensus.cc:686] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0 [term 1 LEADER]: Becoming Leader. State: Replica: 453459e6ea1c47128c7cac11d14303d0, State: Running, Role: LEADER
I20260811 12:47:58.095798 25172 consensus_queue.cc:237] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "453459e6ea1c47128c7cac11d14303d0" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36359 } }
I20260811 12:47:58.096535 25165 sys_catalog.cc:564] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:47:58.098866 25173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "453459e6ea1c47128c7cac11d14303d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "453459e6ea1c47128c7cac11d14303d0" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36359 } } }
I20260811 12:47:58.099563 25173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:58.099298 25174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 453459e6ea1c47128c7cac11d14303d0. Latest consensus state: current_term: 1 leader_uuid: "453459e6ea1c47128c7cac11d14303d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "453459e6ea1c47128c7cac11d14303d0" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36359 } } }
I20260811 12:47:58.100332 25174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0 [sys.catalog]: This master's current role is: LEADER
I20260811 12:47:58.102962 25185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:47:58.107964 25185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:47:58.117017 25185 catalog_manager.cc:1347] Generated new cluster ID: 48451e38d29e421d97b6e812aa4979fd
I20260811 12:47:58.117149 25185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:47:58.151831 25185 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:47:58.153206 25185 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:47:58.165076 25185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0: Generated new TSK 0
I20260811 12:47:58.165962 25185 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.111.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
I20260811 12:47:58.234516 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:36359
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:37915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:47:58.582962 25201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:58.583254 25201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:58.583395 25201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:58.590765 25201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:58.590888 25201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:58.591054 25201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:47:58.601133 25201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:37915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:36359
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:58.602566 25201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:58.603906 25201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:58.617180 25212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:58.617174 25216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:58.617444 25210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:58.617897 25201 server_base.cc:1034] running on GCE node
I20260811 12:47:58.618508 25201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:58.619453 25201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:58.620659 25201 hybrid_clock.cc:648] HybridClock initialized: now 1786452478620604 us; error 47 us; skew 500 ppm
I20260811 12:47:58.620967 25201 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:58 dist-test-slave-f0tn krb5kdc[25027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452478, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:58.624125 25201 init.cc:377] Logged in from keytab as kudu/127.7.111.65@KRBTEST.COM (short username kudu)
I20260811 12:47:58.625615 25201 webserver.cc:460] Webserver started at http://127.7.111.65:38971/ using document root <none> and password file <none>
I20260811 12:47:58.626303 25201 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:58.626425 25201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:58.626739 25201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:58.629762 25201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "4ca8a86b8de44d60aec2779112415898"
format_stamp: "Formatted at 2026-08-11 12:47:58 on dist-test-slave-f0tn"
server_key: "37ba7f3ca74b477bff59167b1c569a0e"
server_key_iv: "0db3461fe5baa7b787fef0e43b4009e8"
server_key_version: "encryptionkey@0"
I20260811 12:47:58.630528 25201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "4ca8a86b8de44d60aec2779112415898"
format_stamp: "Formatted at 2026-08-11 12:47:58 on dist-test-slave-f0tn"
server_key: "37ba7f3ca74b477bff59167b1c569a0e"
server_key_iv: "0db3461fe5baa7b787fef0e43b4009e8"
server_key_version: "encryptionkey@0"
I20260811 12:47:58.635761 25201 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:47:58.639485 25224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:58.640826 25201 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:47:58.641000 25201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "4ca8a86b8de44d60aec2779112415898"
format_stamp: "Formatted at 2026-08-11 12:47:58 on dist-test-slave-f0tn"
server_key: "37ba7f3ca74b477bff59167b1c569a0e"
server_key_iv: "0db3461fe5baa7b787fef0e43b4009e8"
server_key_version: "encryptionkey@0"
I20260811 12:47:58.641197 25201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:58.659318 25201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:58.664058 25201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:58.664330 25201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:58.665593 25201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:58.667068 25201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:58.667146 25201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:58.667258 25201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:58.667313 25201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:58.698523 25201 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:37921
I20260811 12:47:58.698553 25412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:37921 every 8 connection(s)
I20260811 12:47:58.699884 25201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:47:58.705715 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 25201
I20260811 12:47:58.705935 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:47:58.706701 7613 external_mini_cluster.cc:1428] Setting key 1d9055168d616d51d5733c51367cb024
Aug 11 12:47:58 dist-test-slave-f0tn krb5kdc[25027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452478, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:47:58.721549 25417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:58.702550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:53139 (local address 127.7.111.126:36359)
0811 12:47:58.703013 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:58.703019 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:58.704064 (+ 1045us) server_negotiation.cc:407] Connection header received
0811 12:47:58.706021 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:58.706027 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:58.706124 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:58.706252 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:58.707965 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:58.708936 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:58.711158 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:58.711556 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:58.711650 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:58.715557 (+ 3907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:58.715563 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:58.715567 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:58.715607 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:58.718565 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:58.719267 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:58.719272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:58.719276 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:58.719420 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:58.719808 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:58.719812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:58.719815 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:58.720148 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:58.720296 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:58.720950 (+ 654us) server_negotiation.cc:299] Negotiation successful
0811 12:47:58.721164 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":215,"threads_started":1}
I20260811 12:47:58.722427 25420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:58.703125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:36359 (local address 127.7.111.65:53139)
0811 12:47:58.703903 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:58.703943 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:58.705659 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:58.706465 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:58.706473 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:58.707070 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:58.707782 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:58.707809 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:58.709114 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:58.709120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:58.711004 (+ 1884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:58.711021 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:58.711790 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:58.711797 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:58.712092 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:58.712808 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:58.712838 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:58.715320 (+ 2482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:58.718698 (+ 3378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:58.718708 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:58.718723 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:58.719139 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:58.719515 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:58.719519 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:58.719521 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:58.719700 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:58.720259 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:58.720265 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:58.720582 (+ 317us) client_negotiation.cc:769] Sending connection context
0811 12:47:58.720808 (+ 226us) client_negotiation.cc:240] Negotiation successful
0811 12:47:58.721195 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":299,"threads_started":1}
I20260811 12:47:58.724115 25413 heartbeater.cc:347] Connected to a master server at 127.7.111.126:36359
I20260811 12:47:58.724464 25413 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:58.725404 25413 heartbeater.cc:511] Master 127.7.111.126:36359 requested a full tablet report, sending...
I20260811 12:47:58.727919 25106 ts_manager.cc:194] Registered new tserver with Master: 4ca8a86b8de44d60aec2779112415898 (127.7.111.65:37921)
I20260811 12:47:58.729940 25106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:53139
WARNING: no policy specified for kudu/127.7.111.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
I20260811 12:47:58.774607 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:36359
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:37915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:47:59.109565 25426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:59.109840 25426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:59.109977 25426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:59.117183 25426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:59.117292 25426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:59.117458 25426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:47:59.127014 25426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:37915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:36359
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:59.128329 25426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:59.129763 25426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:59.143079 25433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:59.143049 25436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:59.143105 25434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:59.143848 25426 server_base.cc:1034] running on GCE node
I20260811 12:47:59.144538 25426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:59.145617 25426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:59.146845 25426 hybrid_clock.cc:648] HybridClock initialized: now 1786452479146788 us; error 50 us; skew 500 ppm
I20260811 12:47:59.147153 25426 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:59 dist-test-slave-f0tn krb5kdc[25027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452479, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:59.150321 25426 init.cc:377] Logged in from keytab as kudu/127.7.111.66@KRBTEST.COM (short username kudu)
I20260811 12:47:59.151782 25426 webserver.cc:460] Webserver started at http://127.7.111.66:37083/ using document root <none> and password file <none>
I20260811 12:47:59.152470 25426 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:59.152570 25426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:59.152829 25426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:59.155889 25426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "b0a59a13658c4e8d82571b61964ee7d1"
format_stamp: "Formatted at 2026-08-11 12:47:59 on dist-test-slave-f0tn"
server_key: "5ceeeb8e3c9c788431a1079d940bd0f7"
server_key_iv: "162fc0753a41b1c0d93cdd8fb3457cd2"
server_key_version: "encryptionkey@0"
I20260811 12:47:59.156643 25426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "b0a59a13658c4e8d82571b61964ee7d1"
format_stamp: "Formatted at 2026-08-11 12:47:59 on dist-test-slave-f0tn"
server_key: "5ceeeb8e3c9c788431a1079d940bd0f7"
server_key_iv: "162fc0753a41b1c0d93cdd8fb3457cd2"
server_key_version: "encryptionkey@0"
I20260811 12:47:59.161782 25426 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 12:47:59.165210 25444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:59.166584 25426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:47:59.166751 25426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "b0a59a13658c4e8d82571b61964ee7d1"
format_stamp: "Formatted at 2026-08-11 12:47:59 on dist-test-slave-f0tn"
server_key: "5ceeeb8e3c9c788431a1079d940bd0f7"
server_key_iv: "162fc0753a41b1c0d93cdd8fb3457cd2"
server_key_version: "encryptionkey@0"
I20260811 12:47:59.166929 25426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:59.196249 25426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:59.200801 25426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:59.201135 25426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:59.202396 25426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:59.203821 25426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:59.203900 25426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:59.204033 25426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:59.204095 25426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:59.235731 25426 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:35299
I20260811 12:47:59.235780 25656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:35299 every 8 connection(s)
I20260811 12:47:59.237030 25426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:47:59.242746 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 25426
I20260811 12:47:59.242965 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:47:59.243393 7613 external_mini_cluster.cc:1428] Setting key 76c4c1a416b652ae1b8b2db7be21fadd
Aug 11 12:47:59 dist-test-slave-f0tn krb5kdc[25027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452479, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:47:59.257747 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:59.239575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:58443 (local address 127.7.111.126:36359)
0811 12:47:59.240020 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:59.240024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:59.240875 (+ 851us) server_negotiation.cc:407] Connection header received
0811 12:47:59.242911 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:59.242918 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:59.242995 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:59.243115 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:59.244915 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:59.245958 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:59.247520 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:59.248152 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:59.248280 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:59.251699 (+ 3419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:59.251705 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:59.251710 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:59.251747 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:59.254724 (+ 2977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:59.255436 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:59.255441 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:59.255444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:59.255596 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:59.255978 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:59.255983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:59.255985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:59.256315 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:59.256455 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:59.257134 (+ 679us) server_negotiation.cc:299] Negotiation successful
0811 12:47:59.257351 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":237,"threads_started":1}
I20260811 12:47:59.258633 25665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:59.240006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:36359 (local address 127.7.111.66:58443)
0811 12:47:59.240713 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:59.240759 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:59.242557 (+ 1798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:59.243371 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:59.243379 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:59.243966 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:59.244740 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:59.244765 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:59.246107 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:59.246112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:59.247365 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:59.247377 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:59.248351 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:59.248356 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:59.248560 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:59.249212 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:59.249236 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:59.251460 (+ 2224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:59.254880 (+ 3420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:59.254887 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:59.254901 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:59.255322 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:59.255697 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:59.255700 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:59.255702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:59.255884 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:59.256430 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:59.256436 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:59.256774 (+ 338us) client_negotiation.cc:769] Sending connection context
0811 12:47:59.257007 (+ 233us) client_negotiation.cc:240] Negotiation successful
0811 12:47:59.257452 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":276,"threads_started":1}
I20260811 12:47:59.260130 25657 heartbeater.cc:347] Connected to a master server at 127.7.111.126:36359
I20260811 12:47:59.260496 25657 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:59.261257 25657 heartbeater.cc:511] Master 127.7.111.126:36359 requested a full tablet report, sending...
I20260811 12:47:59.262852 25106 ts_manager.cc:194] Registered new tserver with Master: b0a59a13658c4e8d82571b61964ee7d1 (127.7.111.66:35299)
I20260811 12:47:59.263661 25106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:58443
WARNING: no policy specified for kudu/127.7.111.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
I20260811 12:47:59.310433 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:36359
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:37915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:47:59.651939 25670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:47:59.652245 25670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:47:59.652391 25670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:47:59.659999 25670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:47:59.660131 25670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:47:59.660295 25670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:47:59.670279 25670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:37915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:36359
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:47:59.671816 25670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:47:59.673403 25670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:47:59.687323 25682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:59.687491 25681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:47:59.687397 25685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:47:59.688386 25670 server_base.cc:1034] running on GCE node
I20260811 12:47:59.689067 25670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:47:59.690104 25670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:47:59.691336 25670 hybrid_clock.cc:648] HybridClock initialized: now 1786452479691287 us; error 33 us; skew 500 ppm
I20260811 12:47:59.691692 25670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:47:59 dist-test-slave-f0tn krb5kdc[25027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452479, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:47:59.695096 25670 init.cc:377] Logged in from keytab as kudu/127.7.111.67@KRBTEST.COM (short username kudu)
I20260811 12:47:59.696714 25670 webserver.cc:460] Webserver started at http://127.7.111.67:41695/ using document root <none> and password file <none>
I20260811 12:47:59.697508 25670 fs_manager.cc:359] Metadata directory not provided
I20260811 12:47:59.697645 25670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:47:59.697968 25670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:47:59.701081 25670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "7e5007b3a299414ba258f99a4c7ed7bf"
format_stamp: "Formatted at 2026-08-11 12:47:59 on dist-test-slave-f0tn"
server_key: "83c25f77f540c772297ba93ff19cb9ed"
server_key_iv: "32d9fc2f4170e9a545372d1eb66926b2"
server_key_version: "encryptionkey@0"
I20260811 12:47:59.702005 25670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "7e5007b3a299414ba258f99a4c7ed7bf"
format_stamp: "Formatted at 2026-08-11 12:47:59 on dist-test-slave-f0tn"
server_key: "83c25f77f540c772297ba93ff19cb9ed"
server_key_iv: "32d9fc2f4170e9a545372d1eb66926b2"
server_key_version: "encryptionkey@0"
I20260811 12:47:59.707521 25670 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 12:47:59.710994 25695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:59.712251 25670 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:47:59.712404 25670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "7e5007b3a299414ba258f99a4c7ed7bf"
format_stamp: "Formatted at 2026-08-11 12:47:59 on dist-test-slave-f0tn"
server_key: "83c25f77f540c772297ba93ff19cb9ed"
server_key_iv: "32d9fc2f4170e9a545372d1eb66926b2"
server_key_version: "encryptionkey@0"
I20260811 12:47:59.712596 25670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:47:59.733661 25413 heartbeater.cc:503] Master 127.7.111.126:36359 was elected leader, sending a full tablet report...
I20260811 12:47:59.737926 25670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:47:59.741993 25670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:47:59.742264 25670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:47:59.743219 25670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:47:59.744551 25670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:47:59.744632 25670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:59.744750 25670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:47:59.744804 25670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:47:59.776441 25670 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:38679
I20260811 12:47:59.776517 25899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:38679 every 8 connection(s)
I20260811 12:47:59.777743 25670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:47:59.779542 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 25670
I20260811 12:47:59.779711 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:47:59.780192 7613 external_mini_cluster.cc:1428] Setting key a9e8755ddf6aed5803518315dbb693c7
Aug 11 12:47:59 dist-test-slave-f0tn krb5kdc[25027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452479, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:47:59.797518 25904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:59.780291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:40985 (local address 127.7.111.126:36359)
0811 12:47:59.780793 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:59.780798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:59.781839 (+ 1041us) server_negotiation.cc:407] Connection header received
0811 12:47:59.783685 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:59.783689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:59.783762 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:59.783848 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:59.785259 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:59.786135 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:59.787629 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:59.788025 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:59.788129 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:59.791436 (+ 3307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:59.791443 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:59.791447 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:59.791483 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:59.794127 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:59.795105 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:59.795111 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:59.795114 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:59.795270 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:59.795731 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:59.795735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:59.795738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:59.796075 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:59.796230 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:59.796999 (+ 769us) server_negotiation.cc:299] Negotiation successful
0811 12:47:59.797190 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":218,"threads_started":1}
I20260811 12:47:59.798497 25906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:59.780767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:36359 (local address 127.7.111.67:40985)
0811 12:47:59.781657 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:59.781704 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:59.783437 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:59.784036 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:59.784044 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:59.784542 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:47:59.785099 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:59.785116 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:59.786262 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:59.786266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:59.787470 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:59.787483 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:59.788210 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:59.788215 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:59.788410 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:59.789033 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:47:59.789056 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:47:59.791232 (+ 2176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:47:59.794368 (+ 3136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:59.794375 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:59.794395 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:59.794945 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:59.795394 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:47:59.795398 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:47:59.795400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:47:59.795617 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:47:59.796237 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:47:59.796243 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:47:59.796616 (+ 373us) client_negotiation.cc:769] Sending connection context
0811 12:47:59.796892 (+ 276us) client_negotiation.cc:240] Negotiation successful
0811 12:47:59.797313 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":347,"threads_started":1}
I20260811 12:47:59.799810 25901 heartbeater.cc:347] Connected to a master server at 127.7.111.126:36359
I20260811 12:47:59.800091 25901 heartbeater.cc:464] Registering TS with master...
I20260811 12:47:59.800805 25901 heartbeater.cc:511] Master 127.7.111.126:36359 requested a full tablet report, sending...
I20260811 12:47:59.802219 25106 ts_manager.cc:194] Registered new tserver with Master: 7e5007b3a299414ba258f99a4c7ed7bf (127.7.111.67:38679)
I20260811 12:47:59.803067 25106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:40985
I20260811 12:47:59.808689 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:47:59.824815 25904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:59.813231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51494 (local address 127.7.111.126:36359)
0811 12:47:59.813450 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:59.813457 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:59.813860 (+ 403us) server_negotiation.cc:407] Connection header received
0811 12:47:59.813996 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:59.814000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:59.814071 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:59.814146 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:47:59.815545 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:59.816325 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:59.817658 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:59.818041 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:59.818134 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:59.820288 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:47:59.820293 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:47:59.820296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:47:59.820320 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:47:59.822631 (+ 2311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:59.823213 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:59.823216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:59.823218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:59.823328 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:47:59.823618 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:47:59.823621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:47:59.823624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:47:59.823917 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:47:59.824093 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:59.824353 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 12:47:59.824520 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:47:59.828631 25106 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:47:59.835619 25106 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:47:59.858614 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:59.848066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33362 (local address 127.7.111.67:38679)
0811 12:47:59.848913 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:59.848923 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:59.849609 (+ 686us) server_negotiation.cc:407] Connection header received
0811 12:47:59.850008 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:59.850017 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:59.850279 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:59.850469 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:47:59.851322 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:59.852961 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:59.856012 (+ 3051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:59.857486 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:59.857599 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:59.857846 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:59.858108 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 12:47:59.858289 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":467,"threads_started":1}
I20260811 12:47:59.859030 25923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:59.848261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:37921 (local address 127.0.0.1:41052)
0811 12:47:59.850516 (+ 2255us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:59.850533 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:59.850642 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:59.851568 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:59.851573 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:59.851598 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:47:59.852008 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:59.852021 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:59.853830 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:59.853836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:59.856108 (+ 2272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:59.856123 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:59.857876 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:59.857882 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:59.858056 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:59.858087 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:47:59.858155 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 12:47:59.858700 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2135,"mutex_wait_us":390,"thread_start_us":862,"threads_started":1}
I20260811 12:47:59.859050 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:59.848909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41052 (local address 127.7.111.65:37921)
0811 12:47:59.849595 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:59.849604 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:59.850781 (+ 1177us) server_negotiation.cc:407] Connection header received
0811 12:47:59.850884 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:59.850891 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:59.851119 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:59.851295 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:47:59.852197 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:59.853695 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:59.856311 (+ 2616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:59.857642 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:59.858280 (+ 638us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:59.858487 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:59.858552 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 12:47:59.858702 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":1011,"threads_started":1}
I20260811 12:47:59.859274 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:59.847854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:38679 (local address 127.0.0.1:33362)
0811 12:47:59.849470 (+ 1616us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:59.849507 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:59.849765 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:59.850615 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:59.850622 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:59.850655 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:47:59.851069 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:59.851083 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:59.853111 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:59.853116 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:59.855832 (+ 2716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:59.855850 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:59.857695 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:59.857702 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:59.857889 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:59.857930 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 12:47:59.858045 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 12:47:59.858264 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1357,"mutex_wait_us":45,"thread_start_us":2386,"threads_started":1}
I20260811 12:47:59.860131 25932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:59.851091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:35299 (local address 127.0.0.1:53482)
0811 12:47:59.852564 (+ 1473us) negotiation.cc:107] Waiting for socket to connect
0811 12:47:59.852580 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:47:59.852696 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:47:59.853597 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:47:59.853601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:47:59.853626 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:47:59.854148 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:59.854160 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:59.855434 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:59.855438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:59.857440 (+ 2002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:47:59.857456 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:47:59.859424 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:47:59.859430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:47:59.859587 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:59.859618 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:47:59.859707 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 12:47:59.859824 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1368,"thread_start_us":1222,"threads_started":1}
I20260811 12:47:59.860826 25930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:47:59.850827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53482 (local address 127.7.111.66:35299)
0811 12:47:59.851296 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0811 12:47:59.851303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:47:59.852805 (+ 1502us) server_negotiation.cc:407] Connection header received
0811 12:47:59.852905 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:47:59.852912 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:47:59.853130 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:47:59.853313 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:47:59.854315 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:59.855290 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:59.857605 (+ 2315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:47:59.859144 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:47:59.859285 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:47:59.859412 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 12:47:59.860368 (+ 956us) server_negotiation.cc:299] Negotiation successful
0811 12:47:59.860538 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":426,"threads_started":1}
I20260811 12:47:59.863401 25785 tablet_service.cc:1467] Processing CreateTablet for tablet 4860d918f4fc4017badabd198eb980d0 (DEFAULT_TABLE table=test-table [id=d4a5e6dffd0d4fd8a4b2bc65454e5009]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:47:59.863683 25304 tablet_service.cc:1467] Processing CreateTablet for tablet 4860d918f4fc4017badabd198eb980d0 (DEFAULT_TABLE table=test-table [id=d4a5e6dffd0d4fd8a4b2bc65454e5009]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:47:59.864713 25785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4860d918f4fc4017badabd198eb980d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:59.864289 25537 tablet_service.cc:1467] Processing CreateTablet for tablet 4860d918f4fc4017badabd198eb980d0 (DEFAULT_TABLE table=test-table [id=d4a5e6dffd0d4fd8a4b2bc65454e5009]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:47:59.865105 25304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4860d918f4fc4017badabd198eb980d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:59.865600 25537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4860d918f4fc4017badabd198eb980d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:47:59.876116 25934 tablet_bootstrap.cc:492] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf: Bootstrap starting.
I20260811 12:47:59.879323 25934 tablet_bootstrap.cc:654] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:59.880734 25934 log.cc:824] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf: Log is configured to *not* fsync() on all Append() calls
I20260811 12:47:59.880810 25935 tablet_bootstrap.cc:492] T 4860d918f4fc4017badabd198eb980d0 P 4ca8a86b8de44d60aec2779112415898: Bootstrap starting.
I20260811 12:47:59.883579 25937 tablet_bootstrap.cc:492] T 4860d918f4fc4017badabd198eb980d0 P b0a59a13658c4e8d82571b61964ee7d1: Bootstrap starting.
I20260811 12:47:59.884483 25935 tablet_bootstrap.cc:654] T 4860d918f4fc4017badabd198eb980d0 P 4ca8a86b8de44d60aec2779112415898: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:59.885804 25935 log.cc:824] T 4860d918f4fc4017badabd198eb980d0 P 4ca8a86b8de44d60aec2779112415898: Log is configured to *not* fsync() on all Append() calls
I20260811 12:47:59.886937 25937 tablet_bootstrap.cc:654] T 4860d918f4fc4017badabd198eb980d0 P b0a59a13658c4e8d82571b61964ee7d1: Neither blocks nor log segments found. Creating new log.
I20260811 12:47:59.887077 25934 tablet_bootstrap.cc:492] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf: No bootstrap required, opened a new log
I20260811 12:47:59.887334 25934 ts_tablet_manager.cc:1397] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.002s
I20260811 12:47:59.888188 25937 log.cc:824] T 4860d918f4fc4017badabd198eb980d0 P b0a59a13658c4e8d82571b61964ee7d1: Log is configured to *not* fsync() on all Append() calls
I20260811 12:47:59.889026 25935 tablet_bootstrap.cc:492] T 4860d918f4fc4017badabd198eb980d0 P 4ca8a86b8de44d60aec2779112415898: No bootstrap required, opened a new log
I20260811 12:47:59.889318 25935 ts_tablet_manager.cc:1397] T 4860d918f4fc4017badabd198eb980d0 P 4ca8a86b8de44d60aec2779112415898: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 12:47:59.894876 25934 raft_consensus.cc:348] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5007b3a299414ba258f99a4c7ed7bf" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 38679 } } peers { permanent_uuid: "4ca8a86b8de44d60aec2779112415898" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 37921 } } peers { permanent_uuid: "b0a59a13658c4e8d82571b61964ee7d1" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 35299 } }
I20260811 12:47:59.895229 25934 raft_consensus.cc:374] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:59.895329 25934 raft_consensus.cc:729] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e5007b3a299414ba258f99a4c7ed7bf, State: Initialized, Role: FOLLOWER
I20260811 12:47:59.895891 25934 consensus_queue.cc:260] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5007b3a299414ba258f99a4c7ed7bf" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 38679 } } peers { permanent_uuid: "4ca8a86b8de44d60aec2779112415898" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 37921 } } peers { permanent_uuid: "b0a59a13658c4e8d82571b61964ee7d1" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 35299 } }
I20260811 12:47:59.896122 25937 tablet_bootstrap.cc:492] T 4860d918f4fc4017badabd198eb980d0 P b0a59a13658c4e8d82571b61964ee7d1: No bootstrap required, opened a new log
I20260811 12:47:59.896456 25937 ts_tablet_manager.cc:1397] T 4860d918f4fc4017badabd198eb980d0 P b0a59a13658c4e8d82571b61964ee7d1: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.004s
I20260811 12:47:59.896973 25901 heartbeater.cc:503] Master 127.7.111.126:36359 was elected leader, sending a full tablet report...
I20260811 12:47:59.897723 25934 ts_tablet_manager.cc:1428] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf: Time spent starting tablet: real 0.010s user 0.005s sys 0.007s
I20260811 12:47:59.899438 25935 raft_consensus.cc:348] T 4860d918f4fc4017badabd198eb980d0 P 4ca8a86b8de44d60aec2779112415898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5007b3a299414ba258f99a4c7ed7bf" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 38679 } } peers { permanent_uuid: "4ca8a86b8de44d60aec2779112415898" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 37921 } } peers { permanent_uuid: "b0a59a13658c4e8d82571b61964ee7d1" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 35299 } }
I20260811 12:47:59.899885 25935 raft_consensus.cc:374] T 4860d918f4fc4017badabd198eb980d0 P 4ca8a86b8de44d60aec2779112415898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:59.899986 25935 raft_consensus.cc:729] T 4860d918f4fc4017badabd198eb980d0 P 4ca8a86b8de44d60aec2779112415898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ca8a86b8de44d60aec2779112415898, State: Initialized, Role: FOLLOWER
I20260811 12:47:59.900610 25935 consensus_queue.cc:260] T 4860d918f4fc4017badabd198eb980d0 P 4ca8a86b8de44d60aec2779112415898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5007b3a299414ba258f99a4c7ed7bf" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 38679 } } peers { permanent_uuid: "4ca8a86b8de44d60aec2779112415898" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 37921 } } peers { permanent_uuid: "b0a59a13658c4e8d82571b61964ee7d1" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 35299 } }
I20260811 12:47:59.902521 25935 ts_tablet_manager.cc:1428] T 4860d918f4fc4017badabd198eb980d0 P 4ca8a86b8de44d60aec2779112415898: Time spent starting tablet: real 0.013s user 0.010s sys 0.002s
I20260811 12:47:59.906754 25937 raft_consensus.cc:348] T 4860d918f4fc4017badabd198eb980d0 P b0a59a13658c4e8d82571b61964ee7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5007b3a299414ba258f99a4c7ed7bf" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 38679 } } peers { permanent_uuid: "4ca8a86b8de44d60aec2779112415898" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 37921 } } peers { permanent_uuid: "b0a59a13658c4e8d82571b61964ee7d1" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 35299 } }
I20260811 12:47:59.907133 25937 raft_consensus.cc:374] T 4860d918f4fc4017badabd198eb980d0 P b0a59a13658c4e8d82571b61964ee7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:47:59.907244 25937 raft_consensus.cc:729] T 4860d918f4fc4017badabd198eb980d0 P b0a59a13658c4e8d82571b61964ee7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0a59a13658c4e8d82571b61964ee7d1, State: Initialized, Role: FOLLOWER
I20260811 12:47:59.907788 25937 consensus_queue.cc:260] T 4860d918f4fc4017badabd198eb980d0 P b0a59a13658c4e8d82571b61964ee7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5007b3a299414ba258f99a4c7ed7bf" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 38679 } } peers { permanent_uuid: "4ca8a86b8de44d60aec2779112415898" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 37921 } } peers { permanent_uuid: "b0a59a13658c4e8d82571b61964ee7d1" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 35299 } }
I20260811 12:47:59.908840 25657 heartbeater.cc:503] Master 127.7.111.126:36359 was elected leader, sending a full tablet report...
I20260811 12:47:59.909319 25937 ts_tablet_manager.cc:1428] T 4860d918f4fc4017badabd198eb980d0 P b0a59a13658c4e8d82571b61964ee7d1: Time spent starting tablet: real 0.013s user 0.011s sys 0.000s
W20260811 12:47:59.952184 25416 tablet.cc:2369] T 4860d918f4fc4017badabd198eb980d0 P 4ca8a86b8de44d60aec2779112415898: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:47:59.989158 25660 tablet.cc:2369] T 4860d918f4fc4017badabd198eb980d0 P b0a59a13658c4e8d82571b61964ee7d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:48:00.029368 25902 tablet.cc:2369] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:48:00.054757 25943 raft_consensus.cc:482] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:48:00.055099 25943 raft_consensus.cc:504] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5007b3a299414ba258f99a4c7ed7bf" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 38679 } } peers { permanent_uuid: "4ca8a86b8de44d60aec2779112415898" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 37921 } } peers { permanent_uuid: "b0a59a13658c4e8d82571b61964ee7d1" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 35299 } }
I20260811 12:48:00.057140 25943 leader_election.cc:288] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ca8a86b8de44d60aec2779112415898 (127.7.111.65:37921), b0a59a13658c4e8d82571b61964ee7d1 (127.7.111.66:35299)
I20260811 12:48:00.064173 25930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:00.057742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:52081 (local address 127.7.111.66:35299)
0811 12:48:00.058199 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 12:48:00.058205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:48:00.058223 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:48:00.058292 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:48:00.058297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:48:00.058367 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:48:00.058448 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:48:00.058878 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:00.059817 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:00.061947 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:00.063242 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:00.063387 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:00.063460 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 12:48:00.063791 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 12:48:00.063921 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304}
I20260811 12:48:00.064173 25906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:00.057441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:35299 (local address 127.7.111.67:52081)
0811 12:48:00.057784 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 12:48:00.057805 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:48:00.057939 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:48:00.058502 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:48:00.058507 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:48:00.058531 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:48:00.058725 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:48:00.058735 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:48:00.059963 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:48:00.059967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:48:00.061736 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:48:00.061773 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:48:00.063395 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:48:00.063401 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:48:00.063606 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:00.063646 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 12:48:00.063739 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 12:48:00.063881 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":21}
I20260811 12:48:00.064438 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:00.057927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:37113 (local address 127.7.111.65:37921)
0811 12:48:00.058148 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 12:48:00.058154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:48:00.058394 (+ 240us) server_negotiation.cc:407] Connection header received
0811 12:48:00.058668 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:48:00.058684 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:48:00.058759 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:48:00.058849 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:48:00.059432 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:00.060515 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:00.062593 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:00.063462 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:00.063625 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:00.063694 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 12:48:00.064056 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 12:48:00.064196 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 12:48:00.065132 25946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:00.057566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:37921 (local address 127.7.111.67:37113)
0811 12:48:00.058285 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0811 12:48:00.058303 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:48:00.058517 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:48:00.058878 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:48:00.058882 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:48:00.058908 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:48:00.059275 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:48:00.059288 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:48:00.060683 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:48:00.060689 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:48:00.062426 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:48:00.062438 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:48:00.063683 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:48:00.063690 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:48:00.063872 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:00.063904 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:48:00.064617 (+ 713us) client_negotiation.cc:240] Negotiation successful
0811 12:48:00.064765 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":1,"thread_start_us":584,"threads_started":1}
I20260811 12:48:00.065575 25582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4860d918f4fc4017badabd198eb980d0" candidate_uuid: "7e5007b3a299414ba258f99a4c7ed7bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0a59a13658c4e8d82571b61964ee7d1" is_pre_election: true
I20260811 12:48:00.066171 25582 raft_consensus.cc:2454] T 4860d918f4fc4017badabd198eb980d0 P b0a59a13658c4e8d82571b61964ee7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e5007b3a299414ba258f99a4c7ed7bf in term 0.
I20260811 12:48:00.066219 25342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4860d918f4fc4017badabd198eb980d0" candidate_uuid: "7e5007b3a299414ba258f99a4c7ed7bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ca8a86b8de44d60aec2779112415898" is_pre_election: true
I20260811 12:48:00.066782 25342 raft_consensus.cc:2454] T 4860d918f4fc4017badabd198eb980d0 P 4ca8a86b8de44d60aec2779112415898 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e5007b3a299414ba258f99a4c7ed7bf in term 0.
I20260811 12:48:00.066924 25697 leader_election.cc:302] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e5007b3a299414ba258f99a4c7ed7bf, b0a59a13658c4e8d82571b61964ee7d1; no voters:
I20260811 12:48:00.067353 25943 raft_consensus.cc:2781] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:48:00.067524 25943 raft_consensus.cc:482] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:48:00.067649 25943 raft_consensus.cc:3037] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:48:00.069976 25943 raft_consensus.cc:504] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5007b3a299414ba258f99a4c7ed7bf" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 38679 } } peers { permanent_uuid: "4ca8a86b8de44d60aec2779112415898" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 37921 } } peers { permanent_uuid: "b0a59a13658c4e8d82571b61964ee7d1" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 35299 } }
I20260811 12:48:00.070595 25943 leader_election.cc:288] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf [CANDIDATE]: Term 1 election: Requested vote from peers 4ca8a86b8de44d60aec2779112415898 (127.7.111.65:37921), b0a59a13658c4e8d82571b61964ee7d1 (127.7.111.66:35299)
I20260811 12:48:00.071043 25342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4860d918f4fc4017badabd198eb980d0" candidate_uuid: "7e5007b3a299414ba258f99a4c7ed7bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ca8a86b8de44d60aec2779112415898"
I20260811 12:48:00.071221 25342 raft_consensus.cc:3037] T 4860d918f4fc4017badabd198eb980d0 P 4ca8a86b8de44d60aec2779112415898 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:48:00.071165 25582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4860d918f4fc4017badabd198eb980d0" candidate_uuid: "7e5007b3a299414ba258f99a4c7ed7bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0a59a13658c4e8d82571b61964ee7d1"
I20260811 12:48:00.071376 25582 raft_consensus.cc:3037] T 4860d918f4fc4017badabd198eb980d0 P b0a59a13658c4e8d82571b61964ee7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:48:00.073413 25342 raft_consensus.cc:2454] T 4860d918f4fc4017badabd198eb980d0 P 4ca8a86b8de44d60aec2779112415898 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e5007b3a299414ba258f99a4c7ed7bf in term 1.
I20260811 12:48:00.073609 25582 raft_consensus.cc:2454] T 4860d918f4fc4017badabd198eb980d0 P b0a59a13658c4e8d82571b61964ee7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e5007b3a299414ba258f99a4c7ed7bf in term 1.
I20260811 12:48:00.073863 25701 leader_election.cc:302] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ca8a86b8de44d60aec2779112415898, 7e5007b3a299414ba258f99a4c7ed7bf; no voters:
I20260811 12:48:00.074180 25943 raft_consensus.cc:2781] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:48:00.074625 25943 raft_consensus.cc:686] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf [term 1 LEADER]: Becoming Leader. State: Replica: 7e5007b3a299414ba258f99a4c7ed7bf, State: Running, Role: LEADER
I20260811 12:48:00.075119 25943 consensus_queue.cc:237] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5007b3a299414ba258f99a4c7ed7bf" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 38679 } } peers { permanent_uuid: "4ca8a86b8de44d60aec2779112415898" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 37921 } } peers { permanent_uuid: "b0a59a13658c4e8d82571b61964ee7d1" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 35299 } }
I20260811 12:48:00.080070 25106 catalog_manager.cc:5495] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e5007b3a299414ba258f99a4c7ed7bf (127.7.111.67). New cstate: current_term: 1 leader_uuid: "7e5007b3a299414ba258f99a4c7ed7bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5007b3a299414ba258f99a4c7ed7bf" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 38679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ca8a86b8de44d60aec2779112415898" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 37921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0a59a13658c4e8d82571b61964ee7d1" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 35299 } health_report { overall_health: UNKNOWN } } }
I20260811 12:48:00.097088 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:00.091774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33378 (local address 127.7.111.67:38679)
0811 12:48:00.091994 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 12:48:00.092000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:48:00.092018 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:48:00.092162 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:48:00.092166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:48:00.092255 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:48:00.092391 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:48:00.092782 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:00.093594 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:00.094893 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:00.095303 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:00.095414 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:00.095909 (+ 495us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:48:00.096507 (+ 598us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:48:00.096647 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 12:48:00.096753 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 12:48:00.096840 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 12:48:00.105178 25342 raft_consensus.cc:1261] T 4860d918f4fc4017badabd198eb980d0 P 4ca8a86b8de44d60aec2779112415898 [term 1 FOLLOWER]: Refusing update from remote peer 7e5007b3a299414ba258f99a4c7ed7bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:48:00.105180 25582 raft_consensus.cc:1261] T 4860d918f4fc4017badabd198eb980d0 P b0a59a13658c4e8d82571b61964ee7d1 [term 1 FOLLOWER]: Refusing update from remote peer 7e5007b3a299414ba258f99a4c7ed7bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:48:00.106243 25947 consensus_queue.cc:1035] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ca8a86b8de44d60aec2779112415898" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 37921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:48:00.106575 25943 consensus_queue.cc:1035] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0a59a13658c4e8d82571b61964ee7d1" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 35299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:48:00.136618 25103 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:48:00.136979 25103 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:48:00.141039 25103 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 453459e6ea1c47128c7cac11d14303d0: Sending DeleteTablet for 3 replicas of tablet 4860d918f4fc4017badabd198eb980d0
I20260811 12:48:00.142401 25304 tablet_service.cc:1514] Processing DeleteTablet for tablet 4860d918f4fc4017badabd198eb980d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:48:00 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:41052
I20260811 12:48:00.142413 25785 tablet_service.cc:1514] Processing DeleteTablet for tablet 4860d918f4fc4017badabd198eb980d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:48:00 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:33362
I20260811 12:48:00.142401 25537 tablet_service.cc:1514] Processing DeleteTablet for tablet 4860d918f4fc4017badabd198eb980d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:48:00 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:53482
I20260811 12:48:00.145193 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 25201
I20260811 12:48:00.157792 25973 tablet_replica.cc:330] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf: stopping tablet replica
I20260811 12:48:00.158375 25973 raft_consensus.cc:2229] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:48:00.159133 25973 raft_consensus.cc:2258] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:48:00.161780 25973 ts_tablet_manager.cc:1905] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:48:00.169479 25973 ts_tablet_manager.cc:1918] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:48:00.169680 25973 log.cc:1196] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/4860d918f4fc4017badabd198eb980d0
I20260811 12:48:00.170291 25973 ts_tablet_manager.cc:1939] T 4860d918f4fc4017badabd198eb980d0 P 7e5007b3a299414ba258f99a4c7ed7bf: Deleting consensus metadata
I20260811 12:48:00.172277 25083 catalog_manager.cc:4841] TS 7e5007b3a299414ba258f99a4c7ed7bf (127.7.111.67:38679): tablet 4860d918f4fc4017badabd198eb980d0 (table test-table [id=d4a5e6dffd0d4fd8a4b2bc65454e5009]) successfully deleted
I20260811 12:48:00.173943 25967 tablet_replica.cc:330] T 4860d918f4fc4017badabd198eb980d0 P 4ca8a86b8de44d60aec2779112415898: stopping tablet replica
I20260811 12:48:00.174468 25967 raft_consensus.cc:2229] T 4860d918f4fc4017badabd198eb980d0 P 4ca8a86b8de44d60aec2779112415898 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:48:00.175159 25970 tablet_replica.cc:330] T 4860d918f4fc4017badabd198eb980d0 P b0a59a13658c4e8d82571b61964ee7d1: stopping tablet replica
I20260811 12:48:00.175595 25967 raft_consensus.cc:2258] T 4860d918f4fc4017badabd198eb980d0 P 4ca8a86b8de44d60aec2779112415898 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:48:00.175608 25970 raft_consensus.cc:2229] T 4860d918f4fc4017badabd198eb980d0 P b0a59a13658c4e8d82571b61964ee7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:48:00.176116 25970 raft_consensus.cc:2258] T 4860d918f4fc4017badabd198eb980d0 P b0a59a13658c4e8d82571b61964ee7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:48:00.178411 25970 ts_tablet_manager.cc:1905] T 4860d918f4fc4017badabd198eb980d0 P b0a59a13658c4e8d82571b61964ee7d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:48:00.178589 25967 ts_tablet_manager.cc:1905] T 4860d918f4fc4017badabd198eb980d0 P 4ca8a86b8de44d60aec2779112415898: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 12:48:00 dist-test-slave-f0tn krb5kdc[25027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.65@KRBTEST.COM
I20260811 12:48:00.185743 25967 ts_tablet_manager.cc:1918] T 4860d918f4fc4017badabd198eb980d0 P 4ca8a86b8de44d60aec2779112415898: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:48:00.186046 25967 log.cc:1196] T 4860d918f4fc4017badabd198eb980d0 P 4ca8a86b8de44d60aec2779112415898: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/4860d918f4fc4017badabd198eb980d0
I20260811 12:48:00.186066 25970 ts_tablet_manager.cc:1918] T 4860d918f4fc4017badabd198eb980d0 P b0a59a13658c4e8d82571b61964ee7d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:48:00.186259 25970 log.cc:1196] T 4860d918f4fc4017badabd198eb980d0 P b0a59a13658c4e8d82571b61964ee7d1: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/4860d918f4fc4017badabd198eb980d0
I20260811 12:48:00.186699 25967 ts_tablet_manager.cc:1939] T 4860d918f4fc4017badabd198eb980d0 P 4ca8a86b8de44d60aec2779112415898: Deleting consensus metadata
I20260811 12:48:00.186801 25970 ts_tablet_manager.cc:1939] T 4860d918f4fc4017badabd198eb980d0 P b0a59a13658c4e8d82571b61964ee7d1: Deleting consensus metadata
I20260811 12:48:00.188704 25083 catalog_manager.cc:4841] TS b0a59a13658c4e8d82571b61964ee7d1 (127.7.111.66:35299): tablet 4860d918f4fc4017badabd198eb980d0 (table test-table [id=d4a5e6dffd0d4fd8a4b2bc65454e5009]) successfully deleted
I20260811 12:48:00.188711 25085 catalog_manager.cc:4841] TS 4ca8a86b8de44d60aec2779112415898 (127.7.111.65:37921): tablet 4860d918f4fc4017badabd198eb980d0 (table test-table [id=d4a5e6dffd0d4fd8a4b2bc65454e5009]) successfully deleted
I20260811 12:48:00.191300 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:00.146911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41066 (local address 127.7.111.65:37921)
0811 12:48:00.147134 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 12:48:00.147141 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:48:00.171033 (+ 23892us) server_negotiation.cc:407] Connection header received
0811 12:48:00.171599 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:48:00.171607 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:48:00.171725 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:48:00.171803 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:48:00.173985 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:00.175244 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:00.177142 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:00.177775 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:00.177877 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:00.183449 (+ 5572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:48:00.183469 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:48:00.183490 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:48:00.183534 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:48:00.187633 (+ 4099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:48:00.188889 (+ 1256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:48:00.188899 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:48:00.188902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:48:00.189125 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:48:00.189577 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:48:00.189583 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:48:00.189586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:48:00.190054 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:48:00.190406 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0811 12:48:00.190670 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 12:48:00.190910 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 12:48:00.192045 25406 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:48:00.281987 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 25201
I20260811 12:48:00.315891 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 25426
Aug 11 12:48:00 dist-test-slave-f0tn krb5kdc[25027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.66@KRBTEST.COM
I20260811 12:48:00.331378 25930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:00.316715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53484 (local address 127.7.111.66:35299)
0811 12:48:00.316965 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 12:48:00.316971 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:48:00.316992 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:48:00.317058 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:48:00.317064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:48:00.317134 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:48:00.317227 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:48:00.318483 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:00.319226 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:00.320486 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:00.320874 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:00.320981 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:00.325069 (+ 4088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:48:00.325086 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:48:00.325105 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:48:00.325148 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:48:00.328302 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:48:00.329032 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:48:00.329048 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:48:00.329050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:48:00.329187 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:48:00.329736 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:48:00.329741 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:48:00.329743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:48:00.330168 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:48:00.330454 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0811 12:48:00.330802 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 12:48:00.331029 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260811 12:48:00.332111 25652 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:48:00.421671 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 25426
I20260811 12:48:00.448817 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 25670
Aug 11 12:48:00 dist-test-slave-f0tn krb5kdc[25027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.67@KRBTEST.COM
I20260811 12:48:00.466832 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:00.449729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33392 (local address 127.7.111.67:38679)
0811 12:48:00.449977 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 12:48:00.449984 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:48:00.450037 (+ 53us) server_negotiation.cc:407] Connection header received
0811 12:48:00.450112 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:48:00.450118 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:48:00.450222 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:48:00.450343 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:48:00.451652 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:00.452895 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:00.454325 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:00.455003 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:00.455155 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:00.459463 (+ 4308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:48:00.459486 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:48:00.459512 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:48:00.459574 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:48:00.463726 (+ 4152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:48:00.464445 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:48:00.464456 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:48:00.464459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:48:00.464660 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:48:00.465055 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:48:00.465061 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:48:00.465065 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:48:00.465623 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:48:00.465962 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0811 12:48:00.466160 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 12:48:00.466409 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 12:48:00.467749 25894 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:48:00.595606 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 25670
I20260811 12:48:00.622895 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 25055
I20260811 12:48:00.624096 25156 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:48:00.735271 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 25055
2026-08-11T12:48:00Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4722 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:48:00 dist-test-slave-f0tn krb5kdc[25985](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:48:00 dist-test-slave-f0tn krb5kdc[25985](info): set up 2 sockets
Aug 11 12:48:00 dist-test-slave-f0tn krb5kdc[25985](info): commencing operation
krb5kdc: starting...
W20260811 12:48:01.999542 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:48:02 dist-test-slave-f0tn krb5kdc[25985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:48:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:48:02Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.111.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.111.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.111.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/rangeradmin_127.7.111.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.111.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/rangeradmin_127.7.111.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.111.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.111.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.111.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/rangerlookup_127.7.111.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.111.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/rangerlookup_127.7.111.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.111.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.111.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.111.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/HTTP_127.7.111.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.111.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/HTTP_127.7.111.84@KRBTEST.COM.keytab.
I20260811 12:48:02.424057 7613 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskIqQ7YY/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/postgres -l logfile start
2026-08-11 12:48:04.821 UTC [26037] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 12:48:04.821 UTC [26037] LOG: listening on IPv4 address "127.7.111.84", port 45733
2026-08-11 12:48:04.824 UTC [26037] LOG: listening on Unix socket "/tmp/.s.PGSQL.45733"
2026-08-11 12:48:04.847 UTC [26042] LOG: database system was shut down at 2026-08-11 12:48:03 UTC
2026-08-11 12:48:04.851 UTC [26037] LOG: database system is ready to accept connections
I20260811 12:48:05.972434 7613 mini_postgres.cc:96] Postgres bound to 45733
2026-08-11 12:48:06.000 UTC [26057] FATAL: database "slave" does not exist
127.7.111.84:45733 - accepting connections
I20260811 12:48:06.001044 7613 mini_ranger.cc:161] Starting Ranger...
I20260811 12:48:06.035692 7613 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 12:48:06.467674 7613 mini_ranger.cc:87] Created ranger Postgres database
I20260811 12:48:06.467805 7613 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ranger-admin
2026-08-11 12:48:06,917 [I] DB FLAVOR :POSTGRES
2026-08-11 12:48:06,918 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 12:48:06,918 [I] Checking connection..
2026-08-11 12:48:06,918 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:45733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 12:48:07,193 [I] Checking connection passed.
2026-08-11 12:48:07,193 [I] --------- Verifying version history table ---------
2026-08-11 12:48:07,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:45733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:48:07,465 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 12:48:07,465 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:45733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:48:07,730 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 12:48:07,730 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 12:48:07,730 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:45733/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 12:48:07.979 UTC [26170] WARNING: there is no transaction in progress
2026-08-11 12:48:07,988 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 12:48:07,988 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:45733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:48:08,222 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 12:48:08,223 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 12:48:08,223 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:45733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 12:48:08,477 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:45733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 12:48:08,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:45733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-f0tn.c.gcp-upstream.internal','N') ;"
2026-08-11 12:48:08,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:45733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 12:48:09,206 [I] Table x_portal_user does not exist in database ranger
2026-08-11 12:48:09,206 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:45733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 12:48:09,464 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 12:48:09,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:45733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 12:48:09,708 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:45733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 12:48:09,954 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 12:48:09,955 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:45733/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 12:48:10.846 UTC [26366] WARNING: there is no transaction in progress
2026-08-11 12:48:10.861 UTC [26366] WARNING: there is no transaction in progress
2026-08-11 12:48:10.876 UTC [26366] WARNING: there is no transaction in progress
2026-08-11 12:48:10.891 UTC [26366] WARNING: there is no transaction in progress
2026-08-11 12:48:10.905 UTC [26366] WARNING: there is no transaction in progress
2026-08-11 12:48:10.921 UTC [26366] WARNING: there is no transaction in progress
2026-08-11 12:48:10.973 UTC [26366] WARNING: there is no transaction in progress
2026-08-11 12:48:10.979 UTC [26366] WARNING: there is no transaction in progress
2026-08-11 12:48:10.992 UTC [26366] WARNING: there is no transaction in progress
2026-08-11 12:48:11.002 UTC [26366] WARNING: there is no transaction in progress
2026-08-11 12:48:11.012 UTC [26366] WARNING: there is no transaction in progress
2026-08-11 12:48:11.025 UTC [26366] WARNING: there is no transaction in progress
2026-08-11 12:48:11.033 UTC [26366] WARNING: there is no transaction in progress
2026-08-11 12:48:11.039 UTC [26366] WARNING: there is no transaction in progress
2026-08-11 12:48:11.045 UTC [26366] WARNING: there is no transaction in progress
2026-08-11 12:48:11.050 UTC [26366] WARNING: there is no transaction in progress
2026-08-11 12:48:11,611 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 12:48:11,612 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:45733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 12:48:11,848 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 12:48:11,849 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:45733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 12:48:12,107 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 12:48:12,108 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:45733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 12:48:12,350 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 12:48:12,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:45733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 12:48:12,594 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:45733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 12:48:12,854 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:45733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-f0tn.c.gcp-upstream.internal';"
2026-08-11 12:48:13,084 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 12:48:13,084 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:45733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 12:48:13,340 [I] DB_PATCHES have already been applied
I20260811 12:48:13.346957 7613 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ranger-admin:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskIqQ7YY/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 12:48:13.347054 7613 mini_ranger.cc:193] Using host: 127.7.111.84
I20260811 12:48:13.371331 7613 mini_ranger.cc:227] Ranger admin URL: http://127.7.111.84:56581
Aug 11, 2026 12:48:13 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:48:13 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:48:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 12:48:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 12:48:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 12:48:14 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 12:48:14 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 12:48:14.705178 7613 mini_ranger.cc:160] Time spent starting Ranger: real 8.704s user 0.000s sys 0.077s
Aug 11, 2026 12:48:33 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 12:48:33 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 12:48:33 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 12:48:33 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 12:48:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 12:48:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 12:48:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 12:48:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 12:48:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 12:48:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 12:48:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 12:48:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 12:48:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 12:48:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 12:48:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 12:48:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 12:48:33 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 12:48:34 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 12:48:35.209051 7613 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.111.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.111.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.111.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/rangerkms_127.7.111.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.111.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/rangerkms_127.7.111.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.111.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.111.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.111.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/HTTP_127.7.111.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.111.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/HTTP_127.7.111.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 12:48:35 dist-test-slave-f0tn krb5kdc[25985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452515, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 12:48:35.429307 7613 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 12:48:35.456444 7613 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 12:48:36.127225 7613 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 12:48:36.127388 7613 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ranger-kms
I20260811 12:48:36.127489 7613 mini_ranger_kms.cc:227] Using postgres at 127.7.111.84:45733
2026-08-11 12:48:36,874 [I] DB FLAVOR :POSTGRES
2026-08-11 12:48:36,874 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 12:48:36,874 [I] Checking connection
2026-08-11 12:48:36,874 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:45733/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 12:48:37,128 [I] connection success
2026-08-11 12:48:37,129 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 12:48:37,129 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 12:48:37,129 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:45733/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 12:48:37,362 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 12:48:37,363 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 12:48:37,363 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 12:48:37,363 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:45733/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 12:48:37,624 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 12:48:38.082180 7613 mini_ranger_kms.cc:310] Created kms service
I20260811 12:48:38.284862 7613 mini_ranger_kms.cc:326] Created kudu user
I20260811 12:48:38.326776 7613 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 12:48:38 dist-test-slave-f0tn krb5kdc[25985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452515, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
I20260811 12:48:38.746829 7613 mini_ranger_kms.cc:384] Added ranger policy
I20260811 12:48:38.748126 7613 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ranger-kms:/tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 12:48:38.748210 7613 mini_ranger_kms.cc:240] Using host: 127.7.111.84
I20260811 12:48:38.777017 7613 mini_ranger_kms.cc:276] Ranger KMS PID: 26672
I20260811 12:48:38.777240 7613 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.111.84:57609
Aug 11, 2026 12:48:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:48:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:48:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 12:48:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 12:48:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 12:48:40 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 12:48:41 dist-test-slave-f0tn krb5kdc[25985](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786452521, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.111.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:48:41.432139 7613 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.003s user 0.126s sys 0.434s
I20260811 12:48:41.432469 7613 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 12:48:41.432684 7613 mini_ranger_kms.cc:401] 127.7.111.84:57609/kms/v1/keys
Aug 11 12:48:41 dist-test-slave-f0tn krb5kdc[25985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452521, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.111.84@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
Aug 11 12:48:42 dist-test-slave-f0tn krb5kdc[25985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452521, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.111.84@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
Aug 11 12:48:44 dist-test-slave-f0tn krb5kdc[25985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:48:44.505515 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:43917
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:37099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:43917
--ranger_config_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.111.84:57609/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:48:44.919915 26757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:48:44.920264 26757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:48:44.920423 26757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:48:44.928967 26757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:48:44.929177 26757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:48:44.929278 26757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:48:44.929370 26757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:48:44.929509 26757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:48:44.940990 26757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:37099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.111.84:57609/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:43917
--ranger_config_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:43917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:48:44.942538 26757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:48:44.944049 26757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:48:44.957921 26768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:48:44.957880 26772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:48:44.958045 26770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:48:44.958457 26757 server_base.cc:1034] running on GCE node
I20260811 12:48:44.959290 26757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:48:44.960773 26757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:48:44.961978 26757 hybrid_clock.cc:648] HybridClock initialized: now 1786452524961933 us; error 33 us; skew 500 ppm
I20260811 12:48:44.962389 26757 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:48:44 dist-test-slave-f0tn krb5kdc[25985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452524, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:48:44.965692 26757 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:48:44.967154 26757 webserver.cc:460] Webserver started at http://127.7.111.126:40637/ using document root <none> and password file <none>
I20260811 12:48:44.967857 26757 fs_manager.cc:359] Metadata directory not provided
I20260811 12:48:44.968016 26757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:48:44.968391 26757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:48:44 dist-test-slave-f0tn krb5kdc[25985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452524, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
I20260811 12:48:45.029883 26757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "25691b93d3c6456792fc40daf9fd2095"
format_stamp: "Formatted at 2026-08-11 12:48:45 on dist-test-slave-f0tn"
server_key: "1004031a82c048624b1c9adc1689d043"
server_key_iv: "f351d82795aa5108dcd940f95cfff9d4"
server_key_version: "kuduclusterkey@0"
I20260811 12:48:45.030730 26757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "25691b93d3c6456792fc40daf9fd2095"
format_stamp: "Formatted at 2026-08-11 12:48:45 on dist-test-slave-f0tn"
server_key: "1004031a82c048624b1c9adc1689d043"
server_key_iv: "f351d82795aa5108dcd940f95cfff9d4"
server_key_version: "kuduclusterkey@0"
I20260811 12:48:45.036443 26757 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 12:48:45.054886 26787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:48:45.056309 26757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:48:45.056470 26757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "25691b93d3c6456792fc40daf9fd2095"
format_stamp: "Formatted at 2026-08-11 12:48:45 on dist-test-slave-f0tn"
server_key: "1004031a82c048624b1c9adc1689d043"
server_key_iv: "f351d82795aa5108dcd940f95cfff9d4"
server_key_version: "kuduclusterkey@0"
I20260811 12:48:45.056676 26757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:48:45.094980 26757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:48:45.098891 26757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:48:45.099186 26757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:48:45.120273 26757 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:43917
I20260811 12:48:45.120352 26870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:43917 every 8 connection(s)
I20260811 12:48:45.121690 26757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:48:45.125908 26871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:48:45.127231 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 26757
I20260811 12:48:45.127414 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:48:45.128099 7613 external_mini_cluster.cc:1428] Setting key 3a2e2930a8ea62486136b0f63ca3fa69
I20260811 12:48:45.138230 26871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095: Bootstrap starting.
Aug 11 12:48:45 dist-test-slave-f0tn krb5kdc[25985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:48:45.142724 26871 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095: Neither blocks nor log segments found. Creating new log.
I20260811 12:48:45.144088 26871 log.cc:824] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095: Log is configured to *not* fsync() on all Append() calls
I20260811 12:48:45.147493 26871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095: No bootstrap required, opened a new log
I20260811 12:48:45.151736 26875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:45.130175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37870 (local address 127.7.111.126:43917)
0811 12:48:45.131061 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0811 12:48:45.131075 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:48:45.131120 (+ 45us) server_negotiation.cc:407] Connection header received
0811 12:48:45.132909 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:48:45.132932 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:48:45.133296 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:48:45.133631 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:48:45.135195 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:45.136538 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:45.138319 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:45.139044 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:45.139174 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:45.143690 (+ 4516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:48:45.143701 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:48:45.143719 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:48:45.143769 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:48:45.147294 (+ 3525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:48:45.148054 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:48:45.148063 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:48:45.148071 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:48:45.148339 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:48:45.148735 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:48:45.148739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:48:45.148741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:48:45.149307 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:48:45.149651 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0811 12:48:45.150088 (+ 437us) server_negotiation.cc:299] Negotiation successful
0811 12:48:45.150501 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":427,"threads_started":1}
I20260811 12:48:45.156476 26871 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25691b93d3c6456792fc40daf9fd2095" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 43917 } }
I20260811 12:48:45.156836 26871 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:48:45.156937 26871 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25691b93d3c6456792fc40daf9fd2095, State: Initialized, Role: FOLLOWER
I20260811 12:48:45.157603 26871 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25691b93d3c6456792fc40daf9fd2095" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 43917 } }
I20260811 12:48:45.157871 26871 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:48:45.158020 26871 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:48:45.158188 26871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:48:45.161303 26871 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25691b93d3c6456792fc40daf9fd2095" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 43917 } }
I20260811 12:48:45.161909 26871 leader_election.cc:302] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25691b93d3c6456792fc40daf9fd2095; no voters:
I20260811 12:48:45.162817 26871 leader_election.cc:288] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:48:45.162825 26878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:48:45.163757 26878 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095 [term 1 LEADER]: Becoming Leader. State: Replica: 25691b93d3c6456792fc40daf9fd2095, State: Running, Role: LEADER
I20260811 12:48:45.164367 26878 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25691b93d3c6456792fc40daf9fd2095" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 43917 } }
I20260811 12:48:45.165277 26871 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:48:45.167122 26879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25691b93d3c6456792fc40daf9fd2095. Latest consensus state: current_term: 1 leader_uuid: "25691b93d3c6456792fc40daf9fd2095" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25691b93d3c6456792fc40daf9fd2095" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 43917 } } }
I20260811 12:48:45.167694 26879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095 [sys.catalog]: This master's current role is: LEADER
I20260811 12:48:45.167461 26880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25691b93d3c6456792fc40daf9fd2095" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25691b93d3c6456792fc40daf9fd2095" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 43917 } } }
I20260811 12:48:45.168730 26880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095 [sys.catalog]: This master's current role is: LEADER
I20260811 12:48:45.174162 26889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:48:45.180348 26889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:48:45.197304 26871 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 12:48:45.198390 26889 catalog_manager.cc:1347] Generated new cluster ID: d07b9bb846b34f45b273ebb4ca6e8013
I20260811 12:48:45.198559 26889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:48:45.222437 26889 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:48:45.225116 26889 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:48:45.235026 26889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095: Generated new TSK 0
I20260811 12:48:45.235980 26889 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.111.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
I20260811 12:48:46.075052 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.111.84:57609/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:43917
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:37099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:48:46 dist-test-slave-f0tn krb5kdc[25985](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786452526, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 12:48:46.545224 26949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:48:46.545785 26949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:48:46.546075 26949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:48:46.558529 26949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:48:46.558676 26949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:48:46.558869 26949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:48:46.575423 26949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:37099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.111.84:57609/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:43917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:48:46.577260 26949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:48:46.579027 26949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:48:46.601716 26960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:48:46.602614 26961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:48:46.608644 26965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:48:46.609323 26949 server_base.cc:1034] running on GCE node
I20260811 12:48:46.610213 26949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:48:46.613299 26949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:48:46.614607 26949 hybrid_clock.cc:648] HybridClock initialized: now 1786452526614490 us; error 89 us; skew 500 ppm
I20260811 12:48:46.615142 26949 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:48:46 dist-test-slave-f0tn krb5kdc[25985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452526, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:48:46.620224 26949 init.cc:377] Logged in from keytab as kudu/127.7.111.65@KRBTEST.COM (short username kudu)
I20260811 12:48:46.631139 26949 webserver.cc:460] Webserver started at http://127.7.111.65:40419/ using document root <none> and password file <none>
I20260811 12:48:46.632174 26949 fs_manager.cc:359] Metadata directory not provided
I20260811 12:48:46.632351 26949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:48:46.632805 26949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:48:46 dist-test-slave-f0tn krb5kdc[25985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452526, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
I20260811 12:48:46.653214 26949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "bb91ec55f4624bf8b67b90b1f088a823"
format_stamp: "Formatted at 2026-08-11 12:48:46 on dist-test-slave-f0tn"
server_key: "ca14718b587139a0a1dc0fac8405fb8a"
server_key_iv: "a4886e52232f9ba27012245ee0570628"
server_key_version: "kuduclusterkey@0"
I20260811 12:48:46.654486 26949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "bb91ec55f4624bf8b67b90b1f088a823"
format_stamp: "Formatted at 2026-08-11 12:48:46 on dist-test-slave-f0tn"
server_key: "ca14718b587139a0a1dc0fac8405fb8a"
server_key_iv: "a4886e52232f9ba27012245ee0570628"
server_key_version: "kuduclusterkey@0"
I20260811 12:48:46.663122 26949 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.005s
I20260811 12:48:46.695034 26983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:48:46.697031 26949 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:48:46.697350 26949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "bb91ec55f4624bf8b67b90b1f088a823"
format_stamp: "Formatted at 2026-08-11 12:48:46 on dist-test-slave-f0tn"
server_key: "ca14718b587139a0a1dc0fac8405fb8a"
server_key_iv: "a4886e52232f9ba27012245ee0570628"
server_key_version: "kuduclusterkey@0"
I20260811 12:48:46.697789 26949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:48:46.740998 26949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:48:46.746505 26949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:48:46.747062 26949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:48:46.749145 26949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:48:46.754340 26949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:48:46.754518 26949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 12:48:46.754777 26949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:48:46.754923 26949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:48:46 dist-test-slave-f0tn krb5kdc[25985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452526, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
I20260811 12:48:46.939975 26949 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:39947
I20260811 12:48:46.939996 27165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:39947 every 8 connection(s)
I20260811 12:48:46.941449 26949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:48:46.947499 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 26949
I20260811 12:48:46.947707 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:48:46.948360 7613 external_mini_cluster.cc:1428] Setting key e03e5ba1725b138a8bf62586ae2fd1a0
Aug 11 12:48:46 dist-test-slave-f0tn krb5kdc[25985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452526, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:48:46.969115 27174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:46.944810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:43917 (local address 127.7.111.65:53553)
0811 12:48:46.945790 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0811 12:48:46.945848 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 12:48:46.948855 (+ 3007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:48:46.950000 (+ 1145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:48:46.950025 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:48:46.950712 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:48:46.951628 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:48:46.951660 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:48:46.953145 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:48:46.953150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:48:46.954841 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:48:46.954863 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:48:46.956184 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:48:46.956194 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:48:46.956470 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:46.957243 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:48:46.957272 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:48:46.960086 (+ 2814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:48:46.964297 (+ 4211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:48:46.964306 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:48:46.964322 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:48:46.964967 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:48:46.965502 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:48:46.965506 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:48:46.965509 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:48:46.965777 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:48:46.966470 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:48:46.966477 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:48:46.966873 (+ 396us) client_negotiation.cc:769] Sending connection context
0811 12:48:46.967185 (+ 312us) client_negotiation.cc:240] Negotiation successful
0811 12:48:46.967719 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":356,"threads_started":1}
I20260811 12:48:46.970252 27172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:46.944380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:53553 (local address 127.7.111.126:43917)
0811 12:48:46.944919 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0811 12:48:46.944925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:48:46.945993 (+ 1068us) server_negotiation.cc:407] Connection header received
0811 12:48:46.949466 (+ 3473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:48:46.949471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:48:46.949570 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:48:46.949686 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:48:46.951839 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:46.952916 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:46.955003 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:46.955669 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:46.955784 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:46.960318 (+ 4534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:48:46.960324 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:48:46.960327 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:48:46.960364 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:48:46.964115 (+ 3751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:48:46.965117 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:48:46.965122 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:48:46.965125 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:48:46.965357 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:48:46.965913 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:48:46.965917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:48:46.965920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:48:46.966271 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:48:46.966392 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 12:48:46.969652 (+ 3260us) server_negotiation.cc:299] Negotiation successful
0811 12:48:46.969868 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":399,"thread_start_us":258,"threads_started":1}
I20260811 12:48:46.972308 27166 heartbeater.cc:347] Connected to a master server at 127.7.111.126:43917
I20260811 12:48:46.972697 27166 heartbeater.cc:464] Registering TS with master...
I20260811 12:48:46.973644 27166 heartbeater.cc:511] Master 127.7.111.126:43917 requested a full tablet report, sending...
I20260811 12:48:46.976389 26815 ts_manager.cc:194] Registered new tserver with Master: bb91ec55f4624bf8b67b90b1f088a823 (127.7.111.65:39947)
I20260811 12:48:46.978576 26815 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:53553
WARNING: no policy specified for kudu/127.7.111.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
I20260811 12:48:47.036695 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.111.84:57609/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:43917
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:37099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:48:47 dist-test-slave-f0tn krb5kdc[25985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452526, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
W20260811 12:48:47.556053 27177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:48:47.556370 27177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:48:47.556499 27177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:48:47.564671 27177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:48:47.564836 27177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:48:47.565019 27177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:48:47.575850 27177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:37099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.111.84:57609/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:43917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:48:47.577253 27177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:48:47.578653 27177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:48:47.598003 27205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:48:47.598047 27208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:48:47.599751 27203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:48:47.600579 27177 server_base.cc:1034] running on GCE node
I20260811 12:48:47.601233 27177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:48:47.602430 27177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:48:47.603637 27177 hybrid_clock.cc:648] HybridClock initialized: now 1786452527603539 us; error 95 us; skew 500 ppm
I20260811 12:48:47.603979 27177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:48:47 dist-test-slave-f0tn krb5kdc[25985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452527, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:48:47.607386 27177 init.cc:377] Logged in from keytab as kudu/127.7.111.66@KRBTEST.COM (short username kudu)
I20260811 12:48:47.609319 27177 webserver.cc:460] Webserver started at http://127.7.111.66:45925/ using document root <none> and password file <none>
I20260811 12:48:47.610119 27177 fs_manager.cc:359] Metadata directory not provided
I20260811 12:48:47.610284 27177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:48:47.610698 27177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:48:47 dist-test-slave-f0tn krb5kdc[25985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452527, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
I20260811 12:48:47.629340 27177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "5e97f1a2bf594da18310f4fb464b5677"
format_stamp: "Formatted at 2026-08-11 12:48:47 on dist-test-slave-f0tn"
server_key: "e5427d271f9a176fe9d428ad1e20f6a5"
server_key_iv: "29aeba756bb53bb494c6c3445a8752ed"
server_key_version: "kuduclusterkey@0"
I20260811 12:48:47.630661 27177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "5e97f1a2bf594da18310f4fb464b5677"
format_stamp: "Formatted at 2026-08-11 12:48:47 on dist-test-slave-f0tn"
server_key: "e5427d271f9a176fe9d428ad1e20f6a5"
server_key_iv: "29aeba756bb53bb494c6c3445a8752ed"
server_key_version: "kuduclusterkey@0"
I20260811 12:48:47.639294 27177 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260811 12:48:47.659279 27219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:48:47.660938 27177 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 12:48:47.661142 27177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "5e97f1a2bf594da18310f4fb464b5677"
format_stamp: "Formatted at 2026-08-11 12:48:47 on dist-test-slave-f0tn"
server_key: "e5427d271f9a176fe9d428ad1e20f6a5"
server_key_iv: "29aeba756bb53bb494c6c3445a8752ed"
server_key_version: "kuduclusterkey@0"
I20260811 12:48:47.661480 27177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:48:47.685140 27177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:48:47.689935 27177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:48:47.690245 27177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:48:47.691395 27177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:48:47.693105 27177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:48:47.693243 27177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:48:47.693473 27177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:48:47.693549 27177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 12:48:47.740952 27177 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:41271
I20260811 12:48:47.741011 27425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:41271 every 8 connection(s)
I20260811 12:48:47.742398 27177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:48:47.743067 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 27177
I20260811 12:48:47.743245 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:48:47.743848 7613 external_mini_cluster.cc:1428] Setting key cf68570d35b03d45c3fe0287340adc8f
Aug 11 12:48:47 dist-test-slave-f0tn krb5kdc[25985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452527, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:48:47.767086 27429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:47.746729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:45745 (local address 127.7.111.126:43917)
0811 12:48:47.750949 (+ 4220us) server_negotiation.cc:206] Beginning negotiation
0811 12:48:47.750954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:48:47.750969 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:48:47.751014 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:48:47.751017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:48:47.751076 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:48:47.751167 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:48:47.752911 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:47.753707 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:47.755370 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:47.755854 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:47.755969 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:47.759432 (+ 3463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:48:47.759437 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:48:47.759440 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:48:47.759472 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:48:47.762940 (+ 3468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:48:47.763941 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:48:47.763945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:48:47.763948 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:48:47.764103 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:48:47.764655 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:48:47.764660 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:48:47.764662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:48:47.765108 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:48:47.765254 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 12:48:47.766334 (+ 1080us) server_negotiation.cc:299] Negotiation successful
0811 12:48:47.766580 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4108,"thread_start_us":3995,"threads_started":1}
I20260811 12:48:47.769160 27428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:47.746135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:43917 (local address 127.7.111.66:45745)
0811 12:48:47.747048 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0811 12:48:47.747095 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 12:48:47.750128 (+ 3033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:48:47.751466 (+ 1338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:48:47.751475 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:48:47.752011 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:48:47.752738 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:48:47.752758 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:48:47.753855 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:48:47.753859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:48:47.755228 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:48:47.755241 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:48:47.756096 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:48:47.756105 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:48:47.756322 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:47.756949 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:48:47.756977 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:48:47.759223 (+ 2246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:48:47.763102 (+ 3879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:48:47.763112 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:48:47.763129 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:48:47.763809 (+ 680us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:48:47.764209 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:48:47.764214 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:48:47.764218 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:48:47.764535 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:48:47.765246 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:48:47.765254 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:48:47.765730 (+ 476us) client_negotiation.cc:769] Sending connection context
0811 12:48:47.766076 (+ 346us) client_negotiation.cc:240] Negotiation successful
0811 12:48:47.767792 (+ 1716us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":710,"threads_started":1}
I20260811 12:48:47.770812 27426 heartbeater.cc:347] Connected to a master server at 127.7.111.126:43917
I20260811 12:48:47.771147 27426 heartbeater.cc:464] Registering TS with master...
I20260811 12:48:47.771960 27426 heartbeater.cc:511] Master 127.7.111.126:43917 requested a full tablet report, sending...
I20260811 12:48:47.773672 26815 ts_manager.cc:194] Registered new tserver with Master: 5e97f1a2bf594da18310f4fb464b5677 (127.7.111.66:41271)
I20260811 12:48:47.774763 26815 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:45745
WARNING: no policy specified for kudu/127.7.111.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
I20260811 12:48:47.812740 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.111.84:57609/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:43917
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:37099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:48:47.982510 27166 heartbeater.cc:503] Master 127.7.111.126:43917 was elected leader, sending a full tablet report...
W20260811 12:48:48.210029 27432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:48:48.210335 27432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:48:48.210474 27432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:48:48.218397 27432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:48:48.218564 27432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:48:48.218730 27432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:48:48.229351 27432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:37099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.111.84:57609/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:43917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:48:48.230726 27432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:48:48.232087 27432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:48:48.245975 27446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:48:48.246155 27449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:48:48.246166 27447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:48:48.247565 27432 server_base.cc:1034] running on GCE node
I20260811 12:48:48.248252 27432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:48:48.249239 27432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:48:48.250437 27432 hybrid_clock.cc:648] HybridClock initialized: now 1786452528250358 us; error 70 us; skew 500 ppm
I20260811 12:48:48.250757 27432 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:48:48 dist-test-slave-f0tn krb5kdc[25985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452528, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:48:48.256930 27432 init.cc:377] Logged in from keytab as kudu/127.7.111.67@KRBTEST.COM (short username kudu)
I20260811 12:48:48.258404 27432 webserver.cc:460] Webserver started at http://127.7.111.67:37411/ using document root <none> and password file <none>
I20260811 12:48:48.259089 27432 fs_manager.cc:359] Metadata directory not provided
I20260811 12:48:48.259205 27432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:48:48.259496 27432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:48:48 dist-test-slave-f0tn krb5kdc[25985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452528, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
I20260811 12:48:48.274595 27432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "ffd0af403039491f8b41347373637e72"
format_stamp: "Formatted at 2026-08-11 12:48:48 on dist-test-slave-f0tn"
server_key: "0327958080fa56695492d50dfedac48a"
server_key_iv: "3e25d55518fccb902cf82dc037105510"
server_key_version: "kuduclusterkey@0"
I20260811 12:48:48.275391 27432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "ffd0af403039491f8b41347373637e72"
format_stamp: "Formatted at 2026-08-11 12:48:48 on dist-test-slave-f0tn"
server_key: "0327958080fa56695492d50dfedac48a"
server_key_iv: "3e25d55518fccb902cf82dc037105510"
server_key_version: "kuduclusterkey@0"
I20260811 12:48:48.280993 27432 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 12:48:48.296021 27458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:48:48.297402 27432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 12:48:48.297585 27432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "ffd0af403039491f8b41347373637e72"
format_stamp: "Formatted at 2026-08-11 12:48:48 on dist-test-slave-f0tn"
server_key: "0327958080fa56695492d50dfedac48a"
server_key_iv: "3e25d55518fccb902cf82dc037105510"
server_key_version: "kuduclusterkey@0"
I20260811 12:48:48.297783 27432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:48:48.319687 27432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:48:48.323501 27432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:48:48.323796 27432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:48:48.324895 27432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:48:48.326416 27432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:48:48.326494 27432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:48:48.326694 27432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:48:48.326756 27432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:48:48.365880 27432 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:40529
I20260811 12:48:48.365811 27658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:40529 every 8 connection(s)
I20260811 12:48:48.367367 27432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:48:48.372737 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 27432
I20260811 12:48:48.372884 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:48:48.373322 7613 external_mini_cluster.cc:1428] Setting key 290dbfaaaad07c437eb8ff27d4f0eea0
Aug 11 12:48:48 dist-test-slave-f0tn krb5kdc[25985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452528, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:48:48.388430 27661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:48.369916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:59359 (local address 127.7.111.126:43917)
0811 12:48:48.370478 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0811 12:48:48.370483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:48:48.371796 (+ 1313us) server_negotiation.cc:407] Connection header received
0811 12:48:48.373977 (+ 2181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:48:48.373984 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:48:48.374061 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:48:48.374161 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:48:48.376073 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.376890 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.378692 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.379191 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.379291 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:48.382530 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:48:48.382536 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:48:48.382541 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:48:48.382572 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:48:48.385061 (+ 2489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:48:48.385727 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:48:48.385731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:48:48.385733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:48:48.385858 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:48:48.386263 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:48:48.386266 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:48:48.386268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:48:48.386531 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:48:48.386667 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 12:48:48.387544 (+ 877us) server_negotiation.cc:299] Negotiation successful
0811 12:48:48.387883 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":316,"threads_started":1}
I20260811 12:48:48.389358 27662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:48.370782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:43917 (local address 127.7.111.67:59359)
0811 12:48:48.371636 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0811 12:48:48.371673 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:48:48.373668 (+ 1995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:48:48.374420 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:48:48.374428 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:48:48.374856 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:48:48.375873 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:48:48.375907 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.377029 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.377033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:48:48.378530 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:48:48.378570 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.379365 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.379370 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:48:48.379567 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:48.380183 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:48:48.380214 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:48:48.382327 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:48:48.385173 (+ 2846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:48:48.385179 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:48:48.385191 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:48:48.385637 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:48:48.385932 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:48:48.385935 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:48:48.385938 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:48:48.386167 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:48:48.386666 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:48:48.386674 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:48:48.387021 (+ 347us) client_negotiation.cc:769] Sending connection context
0811 12:48:48.387363 (+ 342us) client_negotiation.cc:240] Negotiation successful
0811 12:48:48.387965 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":408,"threads_started":1}
I20260811 12:48:48.390578 27659 heartbeater.cc:347] Connected to a master server at 127.7.111.126:43917
I20260811 12:48:48.390872 27659 heartbeater.cc:464] Registering TS with master...
I20260811 12:48:48.391613 27659 heartbeater.cc:511] Master 127.7.111.126:43917 requested a full tablet report, sending...
I20260811 12:48:48.393014 26815 ts_manager.cc:194] Registered new tserver with Master: ffd0af403039491f8b41347373637e72 (127.7.111.67:40529)
I20260811 12:48:48.393862 26815 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:59359
I20260811 12:48:48.400844 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:48:48.419185 27661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:48.406164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37876 (local address 127.7.111.126:43917)
0811 12:48:48.406376 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 12:48:48.406383 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:48:48.406744 (+ 361us) server_negotiation.cc:407] Connection header received
0811 12:48:48.406910 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:48:48.406913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:48:48.406990 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:48:48.407102 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:48:48.408687 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.409558 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.410929 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.411416 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.411528 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:48.414068 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:48:48.414073 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:48:48.414076 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:48:48.414109 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:48:48.416672 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:48:48.417298 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:48:48.417302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:48:48.417304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:48:48.417469 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:48:48.417829 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:48:48.417834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:48:48.417838 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:48:48.418107 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:48:48.418346 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 12:48:48.418664 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 12:48:48.418886 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:48:48.423267 26815 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:48:48.431270 26815 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:48:48.455308 27685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:48.444618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:39947 (local address 127.0.0.1:57720)
0811 12:48:48.446375 (+ 1757us) negotiation.cc:107] Waiting for socket to connect
0811 12:48:48.446392 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:48:48.446506 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:48:48.447503 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:48:48.447510 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:48:48.447551 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:48:48.448111 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:48:48.448125 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.449805 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.449810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:48:48.452365 (+ 2555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:48:48.452381 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.454199 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.454205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:48:48.454408 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:48.454461 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 12:48:48.454599 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 12:48:48.454786 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1640,"mutex_wait_us":41,"thread_start_us":1528,"threads_started":1}
I20260811 12:48:48.455783 27680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:48.444512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:41271 (local address 127.0.0.1:44792)
0811 12:48:48.445660 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0811 12:48:48.445714 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 12:48:48.445934 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:48:48.448152 (+ 2218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:48:48.448158 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:48:48.448191 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:48:48.448742 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:48:48.448755 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.450834 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.450843 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:48:48.453119 (+ 2276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:48:48.453135 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.454943 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.454952 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:48:48.455168 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:48.455215 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 12:48:48.455288 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 12:48:48.455409 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"mutex_wait_us":17,"thread_start_us":765,"threads_started":1}
I20260811 12:48:48.456182 27687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:48.445247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57720 (local address 127.7.111.65:39947)
0811 12:48:48.446635 (+ 1388us) server_negotiation.cc:206] Beginning negotiation
0811 12:48:48.446644 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:48:48.446676 (+ 32us) server_negotiation.cc:407] Connection header received
0811 12:48:48.446775 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:48:48.446782 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:48:48.447027 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:48:48.447231 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:48:48.448451 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.449647 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.452532 (+ 2885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.453997 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.454123 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:48.454356 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 12:48:48.454660 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 12:48:48.454846 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1210,"thread_start_us":1057,"threads_started":1}
I20260811 12:48:48.456871 27682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:48.444673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44792 (local address 127.7.111.66:41271)
0811 12:48:48.446982 (+ 2309us) server_negotiation.cc:206] Beginning negotiation
0811 12:48:48.446990 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:48:48.447017 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:48:48.447112 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:48:48.447119 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:48:48.447928 (+ 809us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:48:48.448938 (+ 1010us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:48:48.449211 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.450664 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.453290 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.454580 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.456107 (+ 1527us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:48.456303 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 12:48:48.456379 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 12:48:48.456532 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2162,"thread_start_us":2051,"threads_started":1}
I20260811 12:48:48.457573 27688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:48.446985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:40529 (local address 127.0.0.1:54014)
0811 12:48:48.447491 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0811 12:48:48.447506 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:48:48.447652 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:48:48.450621 (+ 2969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:48:48.450625 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:48:48.450655 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:48:48.451392 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:48:48.451406 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.453077 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.453082 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:48:48.454996 (+ 1914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:48:48.455011 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.456838 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.456844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:48:48.457000 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:48.457031 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:48:48.457108 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 12:48:48.457193 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":3,"thread_start_us":830,"threads_started":1}
I20260811 12:48:48.457705 27689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:48.447465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54014 (local address 127.7.111.67:40529)
0811 12:48:48.450037 (+ 2572us) server_negotiation.cc:206] Beginning negotiation
0811 12:48:48.450045 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:48:48.450076 (+ 31us) server_negotiation.cc:407] Connection header received
0811 12:48:48.450181 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:48:48.450188 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:48:48.450399 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:48:48.450599 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:48:48.451635 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.452927 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.455185 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.456635 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.456762 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:48.456970 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 12:48:48.457172 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 12:48:48.457322 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2392,"thread_start_us":2182,"threads_started":1}
I20260811 12:48:48.460464 27313 tablet_service.cc:1467] Processing CreateTablet for tablet 6a3fb7c4298345bdb2080fd8af68eee2 (DEFAULT_TABLE table=test-table [id=1d240701d24d4dc99a7d8f3da1e9e1f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:48:48.460408 27056 tablet_service.cc:1467] Processing CreateTablet for tablet 6a3fb7c4298345bdb2080fd8af68eee2 (DEFAULT_TABLE table=test-table [id=1d240701d24d4dc99a7d8f3da1e9e1f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:48:48.461184 27544 tablet_service.cc:1467] Processing CreateTablet for tablet 6a3fb7c4298345bdb2080fd8af68eee2 (DEFAULT_TABLE table=test-table [id=1d240701d24d4dc99a7d8f3da1e9e1f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:48:48.461859 27313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a3fb7c4298345bdb2080fd8af68eee2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:48:48.462036 27056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a3fb7c4298345bdb2080fd8af68eee2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:48:48.462575 27544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a3fb7c4298345bdb2080fd8af68eee2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:48:48.475551 27690 tablet_bootstrap.cc:492] T 6a3fb7c4298345bdb2080fd8af68eee2 P 5e97f1a2bf594da18310f4fb464b5677: Bootstrap starting.
I20260811 12:48:48.477065 27693 tablet_bootstrap.cc:492] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823: Bootstrap starting.
I20260811 12:48:48.478331 27694 tablet_bootstrap.cc:492] T 6a3fb7c4298345bdb2080fd8af68eee2 P ffd0af403039491f8b41347373637e72: Bootstrap starting.
I20260811 12:48:48.479455 27690 tablet_bootstrap.cc:654] T 6a3fb7c4298345bdb2080fd8af68eee2 P 5e97f1a2bf594da18310f4fb464b5677: Neither blocks nor log segments found. Creating new log.
I20260811 12:48:48.480958 27690 log.cc:824] T 6a3fb7c4298345bdb2080fd8af68eee2 P 5e97f1a2bf594da18310f4fb464b5677: Log is configured to *not* fsync() on all Append() calls
I20260811 12:48:48.481283 27693 tablet_bootstrap.cc:654] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823: Neither blocks nor log segments found. Creating new log.
I20260811 12:48:48.482069 27694 tablet_bootstrap.cc:654] T 6a3fb7c4298345bdb2080fd8af68eee2 P ffd0af403039491f8b41347373637e72: Neither blocks nor log segments found. Creating new log.
I20260811 12:48:48.482822 27693 log.cc:824] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823: Log is configured to *not* fsync() on all Append() calls
I20260811 12:48:48.483296 27694 log.cc:824] T 6a3fb7c4298345bdb2080fd8af68eee2 P ffd0af403039491f8b41347373637e72: Log is configured to *not* fsync() on all Append() calls
I20260811 12:48:48.485365 27690 tablet_bootstrap.cc:492] T 6a3fb7c4298345bdb2080fd8af68eee2 P 5e97f1a2bf594da18310f4fb464b5677: No bootstrap required, opened a new log
I20260811 12:48:48.485735 27690 ts_tablet_manager.cc:1397] T 6a3fb7c4298345bdb2080fd8af68eee2 P 5e97f1a2bf594da18310f4fb464b5677: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.007s
I20260811 12:48:48.486799 27694 tablet_bootstrap.cc:492] T 6a3fb7c4298345bdb2080fd8af68eee2 P ffd0af403039491f8b41347373637e72: No bootstrap required, opened a new log
I20260811 12:48:48.487075 27694 ts_tablet_manager.cc:1397] T 6a3fb7c4298345bdb2080fd8af68eee2 P ffd0af403039491f8b41347373637e72: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 12:48:48.487528 27693 tablet_bootstrap.cc:492] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823: No bootstrap required, opened a new log
I20260811 12:48:48.487807 27693 ts_tablet_manager.cc:1397] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 12:48:48.495940 27693 raft_consensus.cc:348] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e97f1a2bf594da18310f4fb464b5677" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 41271 } } peers { permanent_uuid: "bb91ec55f4624bf8b67b90b1f088a823" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 39947 } } peers { permanent_uuid: "ffd0af403039491f8b41347373637e72" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 40529 } }
I20260811 12:48:48.496346 27693 raft_consensus.cc:374] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:48:48.496456 27693 raft_consensus.cc:729] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb91ec55f4624bf8b67b90b1f088a823, State: Initialized, Role: FOLLOWER
I20260811 12:48:48.496805 27690 raft_consensus.cc:348] T 6a3fb7c4298345bdb2080fd8af68eee2 P 5e97f1a2bf594da18310f4fb464b5677 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e97f1a2bf594da18310f4fb464b5677" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 41271 } } peers { permanent_uuid: "bb91ec55f4624bf8b67b90b1f088a823" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 39947 } } peers { permanent_uuid: "ffd0af403039491f8b41347373637e72" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 40529 } }
I20260811 12:48:48.497053 27693 consensus_queue.cc:260] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e97f1a2bf594da18310f4fb464b5677" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 41271 } } peers { permanent_uuid: "bb91ec55f4624bf8b67b90b1f088a823" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 39947 } } peers { permanent_uuid: "ffd0af403039491f8b41347373637e72" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 40529 } }
I20260811 12:48:48.497222 27690 raft_consensus.cc:374] T 6a3fb7c4298345bdb2080fd8af68eee2 P 5e97f1a2bf594da18310f4fb464b5677 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:48:48.497324 27690 raft_consensus.cc:729] T 6a3fb7c4298345bdb2080fd8af68eee2 P 5e97f1a2bf594da18310f4fb464b5677 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e97f1a2bf594da18310f4fb464b5677, State: Initialized, Role: FOLLOWER
I20260811 12:48:48.497928 27694 raft_consensus.cc:348] T 6a3fb7c4298345bdb2080fd8af68eee2 P ffd0af403039491f8b41347373637e72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e97f1a2bf594da18310f4fb464b5677" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 41271 } } peers { permanent_uuid: "bb91ec55f4624bf8b67b90b1f088a823" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 39947 } } peers { permanent_uuid: "ffd0af403039491f8b41347373637e72" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 40529 } }
I20260811 12:48:48.498080 27690 consensus_queue.cc:260] T 6a3fb7c4298345bdb2080fd8af68eee2 P 5e97f1a2bf594da18310f4fb464b5677 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e97f1a2bf594da18310f4fb464b5677" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 41271 } } peers { permanent_uuid: "bb91ec55f4624bf8b67b90b1f088a823" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 39947 } } peers { permanent_uuid: "ffd0af403039491f8b41347373637e72" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 40529 } }
I20260811 12:48:48.498353 27694 raft_consensus.cc:374] T 6a3fb7c4298345bdb2080fd8af68eee2 P ffd0af403039491f8b41347373637e72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:48:48.498459 27694 raft_consensus.cc:729] T 6a3fb7c4298345bdb2080fd8af68eee2 P ffd0af403039491f8b41347373637e72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffd0af403039491f8b41347373637e72, State: Initialized, Role: FOLLOWER
I20260811 12:48:48.499154 27426 heartbeater.cc:503] Master 127.7.111.126:43917 was elected leader, sending a full tablet report...
I20260811 12:48:48.499044 27694 consensus_queue.cc:260] T 6a3fb7c4298345bdb2080fd8af68eee2 P ffd0af403039491f8b41347373637e72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e97f1a2bf594da18310f4fb464b5677" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 41271 } } peers { permanent_uuid: "bb91ec55f4624bf8b67b90b1f088a823" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 39947 } } peers { permanent_uuid: "ffd0af403039491f8b41347373637e72" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 40529 } }
I20260811 12:48:48.500207 27690 ts_tablet_manager.cc:1428] T 6a3fb7c4298345bdb2080fd8af68eee2 P 5e97f1a2bf594da18310f4fb464b5677: Time spent starting tablet: real 0.014s user 0.004s sys 0.009s
I20260811 12:48:48.501425 27659 heartbeater.cc:503] Master 127.7.111.126:43917 was elected leader, sending a full tablet report...
I20260811 12:48:48.502401 27693 ts_tablet_manager.cc:1428] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260811 12:48:48.502435 27694 ts_tablet_manager.cc:1428] T 6a3fb7c4298345bdb2080fd8af68eee2 P ffd0af403039491f8b41347373637e72: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260811 12:48:48.507392 27698 raft_consensus.cc:482] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:48:48.507637 27698 raft_consensus.cc:504] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e97f1a2bf594da18310f4fb464b5677" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 41271 } } peers { permanent_uuid: "bb91ec55f4624bf8b67b90b1f088a823" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 39947 } } peers { permanent_uuid: "ffd0af403039491f8b41347373637e72" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 40529 } }
I20260811 12:48:48.509562 27698 leader_election.cc:288] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e97f1a2bf594da18310f4fb464b5677 (127.7.111.66:41271), ffd0af403039491f8b41347373637e72 (127.7.111.67:40529)
I20260811 12:48:48.518141 27704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:48.509978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:41271 (local address 127.7.111.65:57907)
0811 12:48:48.510893 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0811 12:48:48.510910 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:48:48.511055 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:48:48.511352 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:48:48.511357 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:48:48.511403 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:48:48.511748 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:48:48.511757 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.513200 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.513204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:48:48.515322 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:48:48.515336 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.517045 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.517054 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:48:48.517319 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:48.517360 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 12:48:48.517537 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 12:48:48.517676 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":806,"mutex_wait_us":2,"thread_start_us":551,"threads_started":1}
I20260811 12:48:48.518220 27682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:48.509774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:57907 (local address 127.7.111.66:41271)
0811 12:48:48.509965 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 12:48:48.509971 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:48:48.511012 (+ 1041us) server_negotiation.cc:407] Connection header received
0811 12:48:48.511167 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:48:48.511171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:48:48.511231 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:48:48.511317 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:48:48.511897 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.513033 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.515476 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.516782 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.516934 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:48.517007 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 12:48:48.517572 (+ 565us) server_negotiation.cc:299] Negotiation successful
0811 12:48:48.518004 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:48:48.519021 27348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a3fb7c4298345bdb2080fd8af68eee2" candidate_uuid: "bb91ec55f4624bf8b67b90b1f088a823" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e97f1a2bf594da18310f4fb464b5677" is_pre_election: true
I20260811 12:48:48.519513 27348 raft_consensus.cc:2454] T 6a3fb7c4298345bdb2080fd8af68eee2 P 5e97f1a2bf594da18310f4fb464b5677 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb91ec55f4624bf8b67b90b1f088a823 in term 0.
I20260811 12:48:48.520223 26984 leader_election.cc:302] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e97f1a2bf594da18310f4fb464b5677, bb91ec55f4624bf8b67b90b1f088a823; no voters:
I20260811 12:48:48.520604 27698 raft_consensus.cc:2781] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:48:48.520802 27698 raft_consensus.cc:482] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:48:48.520973 27698 raft_consensus.cc:3037] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:48:48.521063 27689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:48.509824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:37531 (local address 127.7.111.67:40529)
0811 12:48:48.509994 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 12:48:48.510000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:48:48.510925 (+ 925us) server_negotiation.cc:407] Connection header received
0811 12:48:48.511073 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:48:48.511078 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:48:48.511149 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:48:48.511224 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:48:48.515499 (+ 4275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.516529 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.519139 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.520175 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.520323 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:48.520392 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 12:48:48.520728 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 12:48:48.520839 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 12:48:48.521294 27703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:48.509777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:40529 (local address 127.7.111.65:37531)
0811 12:48:48.510816 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0811 12:48:48.510835 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:48:48.510950 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:48:48.514709 (+ 3759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:48:48.514714 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:48:48.514746 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:48:48.515320 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:48:48.515336 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.516711 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.516717 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:48:48.518992 (+ 2275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:48:48.519005 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.520388 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.520394 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:48:48.520564 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:48.520601 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 12:48:48.520692 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 12:48:48.520820 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"thread_start_us":658,"threads_started":1}
I20260811 12:48:48.522128 27576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a3fb7c4298345bdb2080fd8af68eee2" candidate_uuid: "bb91ec55f4624bf8b67b90b1f088a823" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffd0af403039491f8b41347373637e72" is_pre_election: true
I20260811 12:48:48.522668 27576 raft_consensus.cc:2454] T 6a3fb7c4298345bdb2080fd8af68eee2 P ffd0af403039491f8b41347373637e72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb91ec55f4624bf8b67b90b1f088a823 in term 0.
I20260811 12:48:48.523933 27698 raft_consensus.cc:504] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e97f1a2bf594da18310f4fb464b5677" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 41271 } } peers { permanent_uuid: "bb91ec55f4624bf8b67b90b1f088a823" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 39947 } } peers { permanent_uuid: "ffd0af403039491f8b41347373637e72" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 40529 } }
I20260811 12:48:48.524665 27698 leader_election.cc:288] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823 [CANDIDATE]: Term 1 election: Requested vote from peers 5e97f1a2bf594da18310f4fb464b5677 (127.7.111.66:41271), ffd0af403039491f8b41347373637e72 (127.7.111.67:40529)
I20260811 12:48:48.525061 27348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a3fb7c4298345bdb2080fd8af68eee2" candidate_uuid: "bb91ec55f4624bf8b67b90b1f088a823" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e97f1a2bf594da18310f4fb464b5677"
I20260811 12:48:48.525152 27576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a3fb7c4298345bdb2080fd8af68eee2" candidate_uuid: "bb91ec55f4624bf8b67b90b1f088a823" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffd0af403039491f8b41347373637e72"
I20260811 12:48:48.525272 27348 raft_consensus.cc:3037] T 6a3fb7c4298345bdb2080fd8af68eee2 P 5e97f1a2bf594da18310f4fb464b5677 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:48:48.525352 27576 raft_consensus.cc:3037] T 6a3fb7c4298345bdb2080fd8af68eee2 P ffd0af403039491f8b41347373637e72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:48:48.528646 27576 raft_consensus.cc:2454] T 6a3fb7c4298345bdb2080fd8af68eee2 P ffd0af403039491f8b41347373637e72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb91ec55f4624bf8b67b90b1f088a823 in term 1.
I20260811 12:48:48.528664 27348 raft_consensus.cc:2454] T 6a3fb7c4298345bdb2080fd8af68eee2 P 5e97f1a2bf594da18310f4fb464b5677 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb91ec55f4624bf8b67b90b1f088a823 in term 1.
I20260811 12:48:48.529098 26986 leader_election.cc:302] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb91ec55f4624bf8b67b90b1f088a823, ffd0af403039491f8b41347373637e72; no voters:
I20260811 12:48:48.529438 27698 raft_consensus.cc:2781] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:48:48.530006 27698 raft_consensus.cc:686] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823 [term 1 LEADER]: Becoming Leader. State: Replica: bb91ec55f4624bf8b67b90b1f088a823, State: Running, Role: LEADER
I20260811 12:48:48.530490 27698 consensus_queue.cc:237] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e97f1a2bf594da18310f4fb464b5677" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 41271 } } peers { permanent_uuid: "bb91ec55f4624bf8b67b90b1f088a823" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 39947 } } peers { permanent_uuid: "ffd0af403039491f8b41347373637e72" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 40529 } }
I20260811 12:48:48.535177 26814 catalog_manager.cc:5495] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb91ec55f4624bf8b67b90b1f088a823 (127.7.111.65). New cstate: current_term: 1 leader_uuid: "bb91ec55f4624bf8b67b90b1f088a823" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e97f1a2bf594da18310f4fb464b5677" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 41271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb91ec55f4624bf8b67b90b1f088a823" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ffd0af403039491f8b41347373637e72" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 40529 } health_report { overall_health: UNKNOWN } } }
I20260811 12:48:48.554803 27687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:48.549845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57736 (local address 127.7.111.65:39947)
0811 12:48:48.550048 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 12:48:48.550052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:48:48.550069 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:48:48.550245 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:48:48.550248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:48:48.550326 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:48:48.550456 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:48:48.550796 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.551568 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.552918 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.553308 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.553437 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:48.553825 (+ 388us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:48:48.554343 (+ 518us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:48:48.554455 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 12:48:48.554491 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 12:48:48.554555 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 12:48:48.563468 27348 raft_consensus.cc:1261] T 6a3fb7c4298345bdb2080fd8af68eee2 P 5e97f1a2bf594da18310f4fb464b5677 [term 1 FOLLOWER]: Refusing update from remote peer bb91ec55f4624bf8b67b90b1f088a823: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:48:48.563467 27576 raft_consensus.cc:1261] T 6a3fb7c4298345bdb2080fd8af68eee2 P ffd0af403039491f8b41347373637e72 [term 1 FOLLOWER]: Refusing update from remote peer bb91ec55f4624bf8b67b90b1f088a823: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:48:48.564397 27705 consensus_queue.cc:1035] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e97f1a2bf594da18310f4fb464b5677" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 41271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:48:48.564874 27698 consensus_queue.cc:1035] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffd0af403039491f8b41347373637e72" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 40529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:48:48.577482 27712 mvcc.cc:205] Tried to move back new op lower bound from 7317309556985901056 to 7317309556864487424. Current Snapshot: MvccSnapshot[applied={T|T < 7317309556985901056}]
I20260811 12:48:48.589180 26815 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:48:48.589510 26815 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:48:48.593168 26815 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 25691b93d3c6456792fc40daf9fd2095: Sending DeleteTablet for 3 replicas of tablet 6a3fb7c4298345bdb2080fd8af68eee2
I20260811 12:48:48.594394 27313 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a3fb7c4298345bdb2080fd8af68eee2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:48:48 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:44792
I20260811 12:48:48.594583 27056 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a3fb7c4298345bdb2080fd8af68eee2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:48:48 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:57720
I20260811 12:48:48.594946 27544 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a3fb7c4298345bdb2080fd8af68eee2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:48:48 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:54014
I20260811 12:48:48.599328 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 26949
I20260811 12:48:48.607115 27729 tablet_replica.cc:330] T 6a3fb7c4298345bdb2080fd8af68eee2 P ffd0af403039491f8b41347373637e72: stopping tablet replica
I20260811 12:48:48.607666 27729 raft_consensus.cc:2229] T 6a3fb7c4298345bdb2080fd8af68eee2 P ffd0af403039491f8b41347373637e72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:48:48.608264 27729 raft_consensus.cc:2258] T 6a3fb7c4298345bdb2080fd8af68eee2 P ffd0af403039491f8b41347373637e72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:48:48.613214 27729 ts_tablet_manager.cc:1905] T 6a3fb7c4298345bdb2080fd8af68eee2 P ffd0af403039491f8b41347373637e72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:48:48.620123 27725 tablet_replica.cc:330] T 6a3fb7c4298345bdb2080fd8af68eee2 P 5e97f1a2bf594da18310f4fb464b5677: stopping tablet replica
I20260811 12:48:48.620651 27725 raft_consensus.cc:2229] T 6a3fb7c4298345bdb2080fd8af68eee2 P 5e97f1a2bf594da18310f4fb464b5677 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:48:48.621233 27725 raft_consensus.cc:2258] T 6a3fb7c4298345bdb2080fd8af68eee2 P 5e97f1a2bf594da18310f4fb464b5677 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:48:48.622108 27727 tablet_replica.cc:330] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823: stopping tablet replica
I20260811 12:48:48.622502 27727 raft_consensus.cc:2229] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:48:48.623085 27727 raft_consensus.cc:2258] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 11 12:48:48 dist-test-slave-f0tn krb5kdc[25985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.65@KRBTEST.COM
I20260811 12:48:48.626021 27727 ts_tablet_manager.cc:1905] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:48:48.627296 27729 ts_tablet_manager.cc:1918] T 6a3fb7c4298345bdb2080fd8af68eee2 P ffd0af403039491f8b41347373637e72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:48:48.627462 27729 log.cc:1196] T 6a3fb7c4298345bdb2080fd8af68eee2 P ffd0af403039491f8b41347373637e72: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/6a3fb7c4298345bdb2080fd8af68eee2
I20260811 12:48:48.628002 27729 ts_tablet_manager.cc:1939] T 6a3fb7c4298345bdb2080fd8af68eee2 P ffd0af403039491f8b41347373637e72: Deleting consensus metadata
I20260811 12:48:48.630016 26789 catalog_manager.cc:4841] TS ffd0af403039491f8b41347373637e72 (127.7.111.67:40529): tablet 6a3fb7c4298345bdb2080fd8af68eee2 (table test-table [id=1d240701d24d4dc99a7d8f3da1e9e1f8]) successfully deleted
I20260811 12:48:48.631772 27725 ts_tablet_manager.cc:1905] T 6a3fb7c4298345bdb2080fd8af68eee2 P 5e97f1a2bf594da18310f4fb464b5677: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:48:48.633936 27727 ts_tablet_manager.cc:1918] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:48:48.634143 27727 log.cc:1196] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/6a3fb7c4298345bdb2080fd8af68eee2
I20260811 12:48:48.634756 27727 ts_tablet_manager.cc:1939] T 6a3fb7c4298345bdb2080fd8af68eee2 P bb91ec55f4624bf8b67b90b1f088a823: Deleting consensus metadata
I20260811 12:48:48.636494 27687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:48.601112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57738 (local address 127.7.111.65:39947)
0811 12:48:48.601292 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 12:48:48.601299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:48:48.613955 (+ 12656us) server_negotiation.cc:407] Connection header received
0811 12:48:48.614297 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:48:48.614303 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:48:48.614390 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:48:48.614467 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:48:48.616416 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.617702 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.619746 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.620477 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.620589 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:48.627110 (+ 6521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:48:48.627135 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:48:48.627159 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:48:48.627219 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:48:48.632107 (+ 4888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:48:48.633354 (+ 1247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:48:48.633365 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:48:48.633368 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:48:48.633579 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:48:48.634255 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:48:48.634265 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:48:48.634272 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:48:48.634734 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:48:48.635140 (+ 406us) server_negotiation.cc:1035] Waiting for connection context
0811 12:48:48.635781 (+ 641us) server_negotiation.cc:299] Negotiation successful
0811 12:48:48.636117 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:48:48.636590 26789 catalog_manager.cc:4841] TS bb91ec55f4624bf8b67b90b1f088a823 (127.7.111.65:39947): tablet 6a3fb7c4298345bdb2080fd8af68eee2 (table test-table [id=1d240701d24d4dc99a7d8f3da1e9e1f8]) successfully deleted
I20260811 12:48:48.638350 27159 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:48:48.638551 27725 ts_tablet_manager.cc:1918] T 6a3fb7c4298345bdb2080fd8af68eee2 P 5e97f1a2bf594da18310f4fb464b5677: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:48:48.638784 27725 log.cc:1196] T 6a3fb7c4298345bdb2080fd8af68eee2 P 5e97f1a2bf594da18310f4fb464b5677: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/6a3fb7c4298345bdb2080fd8af68eee2
I20260811 12:48:48.639322 27725 ts_tablet_manager.cc:1939] T 6a3fb7c4298345bdb2080fd8af68eee2 P 5e97f1a2bf594da18310f4fb464b5677: Deleting consensus metadata
I20260811 12:48:48.640906 26788 catalog_manager.cc:4841] TS 5e97f1a2bf594da18310f4fb464b5677 (127.7.111.66:41271): tablet 6a3fb7c4298345bdb2080fd8af68eee2 (table test-table [id=1d240701d24d4dc99a7d8f3da1e9e1f8]) successfully deleted
I20260811 12:48:48.736848 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 26949
I20260811 12:48:48.758868 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 27177
Aug 11 12:48:48 dist-test-slave-f0tn krb5kdc[25985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.66@KRBTEST.COM
I20260811 12:48:48.773867 27682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:48.759807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44796 (local address 127.7.111.66:41271)
0811 12:48:48.760000 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:48:48.760013 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:48:48.760028 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:48:48.760077 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:48:48.760081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:48:48.760138 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:48:48.760221 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:48:48.761599 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.762387 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.763698 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.764098 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.764218 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:48.768479 (+ 4261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:48:48.768491 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:48:48.768508 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:48:48.768544 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:48:48.771175 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:48:48.771793 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:48:48.771800 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:48:48.771803 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:48:48.771936 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:48:48.772286 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:48:48.772290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:48:48.772292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:48:48.772654 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:48:48.772918 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 12:48:48.773278 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 12:48:48.773551 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:48:48.774564 27417 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:48:48.863307 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 27177
I20260811 12:48:48.887398 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 27432
Aug 11 12:48:48 dist-test-slave-f0tn krb5kdc[25985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.67@KRBTEST.COM
I20260811 12:48:48.902137 27689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:48:48.888192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54020 (local address 127.7.111.67:40529)
0811 12:48:48.888377 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 12:48:48.888383 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:48:48.888404 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:48:48.888542 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:48:48.888548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:48:48.888616 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:48:48.888698 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:48:48.890071 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.890798 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.892110 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:48:48.892511 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:48:48.892619 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:48:48.896896 (+ 4277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:48:48.896911 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:48:48.896929 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:48:48.896969 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:48:48.899631 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:48:48.900225 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:48:48.900233 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:48:48.900235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:48:48.900371 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:48:48.900650 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:48:48.900653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:48:48.900654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:48:48.901052 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:48:48.901304 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 12:48:48.901570 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 12:48:48.901791 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:48:48.902743 27652 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:48:48.992321 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 27432
I20260811 12:48:49.014405 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 26757
I20260811 12:48:49.015580 26865 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:48:49.129065 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 26757
I20260811 12:48:49.163883 7613 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 12:48:49.508256 7613 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 12:48:49.508548 7613 mini_ranger.cc:66] Stopping Ranger...
I20260811 12:48:49.863603 7613 mini_ranger.cc:68] Stopped Ranger
2026-08-11 12:48:49.863 UTC [26037] LOG: received smart shutdown request
2026-08-11 12:48:49.867 UTC [26037] LOG: background worker "logical replication launcher" (PID 26054) exited with exit code 1
2026-08-11 12:48:49.867 UTC [26043] LOG: shutting down
2026-08-11 12:48:50.002 UTC [26037] LOG: database system is shut down
2026-08-11T12:48:50Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49366 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:48:50 dist-test-slave-f0tn krb5kdc[27751](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:48:50 dist-test-slave-f0tn krb5kdc[27751](info): set up 2 sockets
Aug 11 12:48:50 dist-test-slave-f0tn krb5kdc[27751](info): commencing operation
krb5kdc: starting...
W20260811 12:48:51.380587 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:48:51 dist-test-slave-f0tn krb5kdc[27751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:48:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:48:51Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.111.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.111.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.111.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/rangeradmin_127.7.111.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.111.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/rangeradmin_127.7.111.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.111.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.111.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.111.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/rangerlookup_127.7.111.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.111.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/rangerlookup_127.7.111.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.111.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.111.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.111.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/HTTP_127.7.111.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.111.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/HTTP_127.7.111.84@KRBTEST.COM.keytab.
I20260811 12:48:51.795747 7613 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskIqQ7YY/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/postgres -l logfile start
2026-08-11 12:48:54.014 UTC [27809] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 12:48:54.014 UTC [27809] LOG: listening on IPv4 address "127.7.111.84", port 36583
2026-08-11 12:48:54.018 UTC [27809] LOG: listening on Unix socket "/tmp/.s.PGSQL.36583"
2026-08-11 12:48:54.038 UTC [27811] LOG: database system was shut down at 2026-08-11 12:48:52 UTC
2026-08-11 12:48:54.042 UTC [27809] LOG: database system is ready to accept connections
I20260811 12:48:55.155732 7613 mini_postgres.cc:96] Postgres bound to 36583
2026-08-11 12:48:55.180 UTC [27827] FATAL: database "slave" does not exist
127.7.111.84:36583 - accepting connections
I20260811 12:48:55.180977 7613 mini_ranger.cc:161] Starting Ranger...
I20260811 12:48:55.210253 7613 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 12:48:55.604353 7613 mini_ranger.cc:87] Created ranger Postgres database
I20260811 12:48:55.604475 7613 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ranger-admin
2026-08-11 12:48:55,877 [I] DB FLAVOR :POSTGRES
2026-08-11 12:48:55,878 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 12:48:55,878 [I] Checking connection..
2026-08-11 12:48:55,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:36583/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 12:48:56,139 [I] Checking connection passed.
2026-08-11 12:48:56,139 [I] --------- Verifying version history table ---------
2026-08-11 12:48:56,139 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:36583/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:48:56,384 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 12:48:56,385 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:36583/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:48:56,643 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 12:48:56,643 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 12:48:56,643 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:36583/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 12:48:56.882 UTC [27937] WARNING: there is no transaction in progress
2026-08-11 12:48:56,890 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 12:48:56,891 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:36583/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:48:57,139 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 12:48:57,140 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 12:48:57,140 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:36583/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 12:48:57,388 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:36583/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 12:48:57,631 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:36583/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-f0tn.c.gcp-upstream.internal','N') ;"
2026-08-11 12:48:57,882 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:36583/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 12:48:58,140 [I] Table x_portal_user does not exist in database ranger
2026-08-11 12:48:58,140 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:36583/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 12:48:58,404 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 12:48:58,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:36583/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 12:48:58,638 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:36583/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 12:48:58,885 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 12:48:58,885 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:36583/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 12:48:59.780 UTC [28130] WARNING: there is no transaction in progress
2026-08-11 12:48:59.797 UTC [28130] WARNING: there is no transaction in progress
2026-08-11 12:48:59.812 UTC [28130] WARNING: there is no transaction in progress
2026-08-11 12:48:59.826 UTC [28130] WARNING: there is no transaction in progress
2026-08-11 12:48:59.840 UTC [28130] WARNING: there is no transaction in progress
2026-08-11 12:48:59.858 UTC [28130] WARNING: there is no transaction in progress
2026-08-11 12:48:59.912 UTC [28130] WARNING: there is no transaction in progress
2026-08-11 12:48:59.917 UTC [28130] WARNING: there is no transaction in progress
2026-08-11 12:48:59.929 UTC [28130] WARNING: there is no transaction in progress
2026-08-11 12:48:59.938 UTC [28130] WARNING: there is no transaction in progress
2026-08-11 12:48:59.948 UTC [28130] WARNING: there is no transaction in progress
2026-08-11 12:48:59.961 UTC [28130] WARNING: there is no transaction in progress
2026-08-11 12:48:59.971 UTC [28130] WARNING: there is no transaction in progress
2026-08-11 12:48:59.977 UTC [28130] WARNING: there is no transaction in progress
2026-08-11 12:48:59.984 UTC [28130] WARNING: there is no transaction in progress
2026-08-11 12:48:59.991 UTC [28130] WARNING: there is no transaction in progress
2026-08-11 12:49:00,536 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 12:49:00,536 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:36583/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 12:49:00,776 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 12:49:00,777 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:36583/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 12:49:01,037 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 12:49:01,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:36583/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 12:49:01,279 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 12:49:01,279 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:36583/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 12:49:01,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:36583/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 12:49:01,767 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:36583/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-f0tn.c.gcp-upstream.internal';"
2026-08-11 12:49:02,018 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 12:49:02,018 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:36583/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 12:49:02,257 [I] DB_PATCHES have already been applied
I20260811 12:49:02.262907 7613 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ranger-admin:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskIqQ7YY/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 12:49:02.263000 7613 mini_ranger.cc:193] Using host: 127.7.111.84
I20260811 12:49:02.285684 7613 mini_ranger.cc:227] Ranger admin URL: http://127.7.111.84:57243
Aug 11, 2026 12:49:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:49:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:49:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 12:49:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 12:49:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 12:49:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 12:49:03 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 12:49:03.579063 7613 mini_ranger.cc:160] Time spent starting Ranger: real 8.398s user 0.004s sys 0.064s
Aug 11, 2026 12:49:22 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 12:49:22 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 12:49:22 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 12:49:22 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 12:49:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 12:49:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 12:49:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 12:49:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 12:49:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 12:49:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 12:49:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 12:49:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 12:49:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 12:49:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 12:49:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 12:49:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 12:49:22 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 12:49:23 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 12:49:23.948064 7613 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.111.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.111.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.111.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/rangerkms_127.7.111.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.111.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/rangerkms_127.7.111.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.111.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.111.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.111.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/HTTP_127.7.111.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.111.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/HTTP_127.7.111.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 12:49:24 dist-test-slave-f0tn krb5kdc[27751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452564, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 12:49:24.182412 7613 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 12:49:24.210757 7613 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 12:49:24.905010 7613 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 12:49:24.905134 7613 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ranger-kms
I20260811 12:49:24.905179 7613 mini_ranger_kms.cc:227] Using postgres at 127.7.111.84:36583
2026-08-11 12:49:25,263 [I] DB FLAVOR :POSTGRES
2026-08-11 12:49:25,263 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 12:49:25,263 [I] Checking connection
2026-08-11 12:49:25,263 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:36583/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 12:49:25,503 [I] connection success
2026-08-11 12:49:25,503 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 12:49:25,503 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 12:49:25,503 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:36583/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 12:49:25,774 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 12:49:25,774 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 12:49:25,774 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 12:49:25,774 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.111.84:36583/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 12:49:26,037 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 12:49:26.512373 7613 mini_ranger_kms.cc:310] Created kms service
I20260811 12:49:26.690855 7613 mini_ranger_kms.cc:326] Created kudu user
I20260811 12:49:26.734323 7613 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 12:49:26 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452564, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
I20260811 12:49:27.150904 7613 mini_ranger_kms.cc:384] Added ranger policy
I20260811 12:49:27.151240 7613 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ranger-kms:/tmp/dist-test-taskIqQ7YY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskIqQ7YY/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 12:49:27.151301 7613 mini_ranger_kms.cc:240] Using host: 127.7.111.84
I20260811 12:49:27.175499 7613 mini_ranger_kms.cc:276] Ranger KMS PID: 28429
I20260811 12:49:27.175601 7613 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.111.84:46741
Aug 11, 2026 12:49:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:49:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:49:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 12:49:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 12:49:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 12:49:28 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 12:49:29 dist-test-slave-f0tn krb5kdc[27751](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786452569, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.111.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:29.798147 7613 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.616s user 0.052s sys 0.354s
I20260811 12:49:29.798403 7613 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 12:49:29.798599 7613 mini_ranger_kms.cc:401] 127.7.111.84:46741/kms/v1/keys
Aug 11 12:49:29 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452569, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.111.84@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
Aug 11 12:49:30 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452569, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.111.84@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
Aug 11 12:49:32 dist-test-slave-f0tn krb5kdc[27751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:49:32.758078 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:36943
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:41997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:36943
--ranger_config_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.111.84:46741/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:49:33.096536 28510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:33.096822 28510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:33.096930 28510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:49:33.104239 28510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:49:33.104352 28510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:33.104405 28510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:33.104460 28510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:49:33.104508 28510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:49:33.114717 28510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:41997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.111.84:46741/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:36943
--ranger_config_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:36943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:33.115974 28510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:33.117410 28510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:33.129448 28521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:33.129510 28517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:33.129781 28510 server_base.cc:1034] running on GCE node
W20260811 12:49:33.129426 28526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:33.130721 28510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:33.132164 28510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:33.133396 28510 hybrid_clock.cc:648] HybridClock initialized: now 1786452573133327 us; error 36 us; skew 500 ppm
I20260811 12:49:33.133740 28510 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:33 dist-test-slave-f0tn krb5kdc[27751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452573, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:33.141938 28510 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:49:33.143477 28510 webserver.cc:460] Webserver started at http://127.7.111.126:41441/ using document root <none> and password file <none>
I20260811 12:49:33.144173 28510 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:33.144284 28510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:49:33.144605 28510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:49:33 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452573, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
I20260811 12:49:33.204156 28510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "c42b56beae984aeab6fda7f7b76d8d96"
format_stamp: "Formatted at 2026-08-11 12:49:33 on dist-test-slave-f0tn"
server_key: "f3d08d23d0043107e6ff6d954af8e621"
server_key_iv: "02ad7a2be42dd1dd9d01ef78376008ee"
server_key_version: "kuduclusterkey@0"
I20260811 12:49:33.205371 28510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "c42b56beae984aeab6fda7f7b76d8d96"
format_stamp: "Formatted at 2026-08-11 12:49:33 on dist-test-slave-f0tn"
server_key: "f3d08d23d0043107e6ff6d954af8e621"
server_key_iv: "02ad7a2be42dd1dd9d01ef78376008ee"
server_key_version: "kuduclusterkey@0"
I20260811 12:49:33.213058 28510 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 12:49:33.234002 28536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:33.235791 28510 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260811 12:49:33.235994 28510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "c42b56beae984aeab6fda7f7b76d8d96"
format_stamp: "Formatted at 2026-08-11 12:49:33 on dist-test-slave-f0tn"
server_key: "f3d08d23d0043107e6ff6d954af8e621"
server_key_iv: "02ad7a2be42dd1dd9d01ef78376008ee"
server_key_version: "kuduclusterkey@0"
I20260811 12:49:33.236263 28510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:33.261154 28510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:33.264825 28510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:33.265121 28510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:33.282297 28510 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:36943
I20260811 12:49:33.282330 28623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:36943 every 8 connection(s)
I20260811 12:49:33.283650 28510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:49:33.287405 28624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:49:33.289628 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 28510
I20260811 12:49:33.289839 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:49:33.290652 7613 external_mini_cluster.cc:1428] Setting key d9faa709fa2e1b2dccd547bf60d2cc0b
I20260811 12:49:33.300823 28624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96: Bootstrap starting.
I20260811 12:49:33.304453 28624 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96: Neither blocks nor log segments found. Creating new log.
Aug 11 12:49:33 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:49:33.305639 28624 log.cc:824] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96: Log is configured to *not* fsync() on all Append() calls
I20260811 12:49:33.308770 28624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96: No bootstrap required, opened a new log
I20260811 12:49:33.313962 28630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:33.292769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57258 (local address 127.7.111.126:36943)
0811 12:49:33.293590 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:33.293606 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:33.293669 (+ 63us) server_negotiation.cc:407] Connection header received
0811 12:49:33.295952 (+ 2283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:33.295981 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:33.296464 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:33.296902 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:33.298216 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:33.300124 (+ 1908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:33.301534 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:33.302564 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:33.302752 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:33.306831 (+ 4079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:33.306842 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:33.306861 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:33.306910 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:33.309884 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:33.310742 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:33.310749 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:33.310755 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:33.310922 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:33.311329 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:33.311332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:33.311337 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:33.311870 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:33.312093 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:33.312532 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 12:49:33.312877 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":337,"threads_started":1}
I20260811 12:49:33.317163 28624 raft_consensus.cc:348] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42b56beae984aeab6fda7f7b76d8d96" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36943 } }
I20260811 12:49:33.317571 28624 raft_consensus.cc:374] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:49:33.317682 28624 raft_consensus.cc:729] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c42b56beae984aeab6fda7f7b76d8d96, State: Initialized, Role: FOLLOWER
I20260811 12:49:33.318230 28624 consensus_queue.cc:260] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42b56beae984aeab6fda7f7b76d8d96" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36943 } }
I20260811 12:49:33.318471 28624 raft_consensus.cc:388] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:49:33.318606 28624 raft_consensus.cc:482] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:49:33.318782 28624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:49:33.321166 28624 raft_consensus.cc:504] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42b56beae984aeab6fda7f7b76d8d96" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36943 } }
I20260811 12:49:33.321655 28624 leader_election.cc:302] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c42b56beae984aeab6fda7f7b76d8d96; no voters:
I20260811 12:49:33.322201 28624 leader_election.cc:288] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:49:33.322327 28632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:49:33.322939 28632 raft_consensus.cc:686] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [term 1 LEADER]: Becoming Leader. State: Replica: c42b56beae984aeab6fda7f7b76d8d96, State: Running, Role: LEADER
I20260811 12:49:33.323633 28632 consensus_queue.cc:237] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42b56beae984aeab6fda7f7b76d8d96" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36943 } }
I20260811 12:49:33.324222 28624 sys_catalog.cc:564] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:49:33.326514 28635 sys_catalog.cc:455] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c42b56beae984aeab6fda7f7b76d8d96. Latest consensus state: current_term: 1 leader_uuid: "c42b56beae984aeab6fda7f7b76d8d96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42b56beae984aeab6fda7f7b76d8d96" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36943 } } }
I20260811 12:49:33.326930 28635 sys_catalog.cc:458] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [sys.catalog]: This master's current role is: LEADER
I20260811 12:49:33.326730 28633 sys_catalog.cc:455] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c42b56beae984aeab6fda7f7b76d8d96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42b56beae984aeab6fda7f7b76d8d96" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36943 } } }
I20260811 12:49:33.327025 28633 sys_catalog.cc:458] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [sys.catalog]: This master's current role is: LEADER
I20260811 12:49:33.328788 28644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:49:33.333819 28644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:49:33.335002 28624 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 12:49:33.356112 28644 catalog_manager.cc:1347] Generated new cluster ID: e6a521c5f42c4f6d8e4337bc82819bc6
I20260811 12:49:33.356261 28644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:49:33.368448 28644 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:49:33.370417 28644 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:49:33.389933 28644 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96: Generated new TSK 0
I20260811 12:49:33.391106 28644 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.111.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
I20260811 12:49:34.163010 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.111.84:46741/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:36943
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:41997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:49:34 dist-test-slave-f0tn krb5kdc[27751](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786452574, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 12:49:34.576545 28705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:34.576850 28705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:34.577028 28705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:49:34.584906 28705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:34.585035 28705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:34.585196 28705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:49:34.595336 28705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:41997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.111.84:46741/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:36943
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:34.596864 28705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:34.598289 28705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:34.614675 28729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:34.620855 28730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:34.628557 28734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:34.628872 28705 server_base.cc:1034] running on GCE node
I20260811 12:49:34.629848 28705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:34.630985 28705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:34.632283 28705 hybrid_clock.cc:648] HybridClock initialized: now 1786452574632207 us; error 63 us; skew 500 ppm
I20260811 12:49:34.632650 28705 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:34 dist-test-slave-f0tn krb5kdc[27751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452574, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:34.636868 28705 init.cc:377] Logged in from keytab as kudu/127.7.111.65@KRBTEST.COM (short username kudu)
I20260811 12:49:34.642405 28705 webserver.cc:460] Webserver started at http://127.7.111.65:42857/ using document root <none> and password file <none>
I20260811 12:49:34.643318 28705 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:34.643539 28705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:49:34.643980 28705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:49:34 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452574, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
I20260811 12:49:34.667209 28705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "25f13b7984a7401ab3623c90a9efa2b0"
format_stamp: "Formatted at 2026-08-11 12:49:34 on dist-test-slave-f0tn"
server_key: "5247c030a1565f7375605328e827a668"
server_key_iv: "880e64e202835566dff37cd8a80a8451"
server_key_version: "kuduclusterkey@0"
I20260811 12:49:34.669410 28705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "25f13b7984a7401ab3623c90a9efa2b0"
format_stamp: "Formatted at 2026-08-11 12:49:34 on dist-test-slave-f0tn"
server_key: "5247c030a1565f7375605328e827a668"
server_key_iv: "880e64e202835566dff37cd8a80a8451"
server_key_version: "kuduclusterkey@0"
I20260811 12:49:34.675844 28705 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.002s
I20260811 12:49:34.705634 28747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:34.707957 28705 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.005s sys 0.001s
I20260811 12:49:34.708307 28705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "25f13b7984a7401ab3623c90a9efa2b0"
format_stamp: "Formatted at 2026-08-11 12:49:34 on dist-test-slave-f0tn"
server_key: "5247c030a1565f7375605328e827a668"
server_key_iv: "880e64e202835566dff37cd8a80a8451"
server_key_version: "kuduclusterkey@0"
I20260811 12:49:34.708742 28705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:34.766530 28705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:34.770766 28705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:34.771072 28705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:34.772121 28705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:49:34.773653 28705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:49:34.773734 28705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:34.773852 28705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:49:34.773908 28705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:34.864174 28705 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:40037
I20260811 12:49:34.864228 28954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:40037 every 8 connection(s)
I20260811 12:49:34.866149 28705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:49:34.871011 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 28705
I20260811 12:49:34.871198 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:49:34.871913 7613 external_mini_cluster.cc:1428] Setting key 786dea1a8b7c75595f4a7902c20d8c42
Aug 11 12:49:34 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452574, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:49:34.938673 28959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:34.870411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:41801 (local address 127.7.111.126:36943)
0811 12:49:34.897141 (+ 26730us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:34.897148 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:34.913433 (+ 16285us) server_negotiation.cc:407] Connection header received
0811 12:49:34.917434 (+ 4001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:34.917441 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:34.917536 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:34.917617 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:34.920767 (+ 3150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:34.922070 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:34.924744 (+ 2674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:34.925484 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:34.925619 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:34.930612 (+ 4993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:34.930619 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:34.930625 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:34.930688 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:34.934419 (+ 3731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:34.935474 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:34.935480 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:34.935484 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:34.935671 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:34.936270 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:34.936276 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:34.936280 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:34.936683 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:34.936821 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:34.937962 (+ 1141us) server_negotiation.cc:299] Negotiation successful
0811 12:49:34.938203 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26572,"thread_start_us":26742,"threads_started":1}
I20260811 12:49:34.941483 28958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:34.869781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:36943 (local address 127.7.111.65:41801)
0811 12:49:34.910308 (+ 40527us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:34.910387 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:34.913177 (+ 2790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:34.918216 (+ 5039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:34.918228 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:34.918933 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:49:34.919761 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:34.919785 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:34.922313 (+ 2528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:34.922320 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:34.924516 (+ 2196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:34.924554 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:34.926031 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:34.926041 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:34.926360 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:34.927233 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:49:34.927269 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:49:34.930299 (+ 3030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:49:34.934666 (+ 4367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:34.934677 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:34.934696 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:34.935314 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:34.935839 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:34.935844 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:34.935848 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:34.936120 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:34.936940 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:49:34.936949 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:49:34.937441 (+ 492us) client_negotiation.cc:769] Sending connection context
0811 12:49:34.939297 (+ 1856us) client_negotiation.cc:240] Negotiation successful
0811 12:49:34.939881 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40098,"thread_start_us":46400,"threads_started":1}
I20260811 12:49:34.943696 28955 heartbeater.cc:347] Connected to a master server at 127.7.111.126:36943
I20260811 12:49:34.944028 28955 heartbeater.cc:464] Registering TS with master...
I20260811 12:49:34.944834 28955 heartbeater.cc:511] Master 127.7.111.126:36943 requested a full tablet report, sending...
I20260811 12:49:34.947266 28563 ts_manager.cc:194] Registered new tserver with Master: 25f13b7984a7401ab3623c90a9efa2b0 (127.7.111.65:40037)
I20260811 12:49:34.949162 28563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:41801
WARNING: no policy specified for kudu/127.7.111.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.66@KRBTEST.COM" created.
Aug 11 12:49:34 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452574, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
W20260811 12:49:35.009627 7613 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.7.111.66: real 0.110s user 0.000s sys 0.026s
I20260811 12:49:35.010684 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.111.84:46741/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:36943
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:41997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:49:35 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452574, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
W20260811 12:49:35.462443 28962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:35.462896 28962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:35.463081 28962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:49:35.474591 28962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:35.474815 28962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:35.475080 28962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:49:35.490695 28962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:41997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.111.84:46741/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:36943
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:35.492784 28962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:35.494707 28962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:35.528049 28976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:35.531822 28980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:35.532521 28984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:35.539238 28962 server_base.cc:1034] running on GCE node
I20260811 12:49:35.540073 28962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:35.545725 28962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:35.547094 28962 hybrid_clock.cc:648] HybridClock initialized: now 1786452575546965 us; error 124 us; skew 500 ppm
I20260811 12:49:35.547536 28962 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:35 dist-test-slave-f0tn krb5kdc[27751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452575, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:35.552038 28962 init.cc:377] Logged in from keytab as kudu/127.7.111.66@KRBTEST.COM (short username kudu)
I20260811 12:49:35.569747 28962 webserver.cc:460] Webserver started at http://127.7.111.66:34053/ using document root <none> and password file <none>
I20260811 12:49:35.570993 28962 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:35.571254 28962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:49:35.571736 28962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:49:35 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452575, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
I20260811 12:49:35.592875 28962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "192e00576a964e7aa82d94d918b9e5ef"
format_stamp: "Formatted at 2026-08-11 12:49:35 on dist-test-slave-f0tn"
server_key: "39897227bca9c6fd025515936c895403"
server_key_iv: "6e88d52cd5ad902a8efbc9458b250a84"
server_key_version: "kuduclusterkey@0"
I20260811 12:49:35.595310 28962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "192e00576a964e7aa82d94d918b9e5ef"
format_stamp: "Formatted at 2026-08-11 12:49:35 on dist-test-slave-f0tn"
server_key: "39897227bca9c6fd025515936c895403"
server_key_iv: "6e88d52cd5ad902a8efbc9458b250a84"
server_key_version: "kuduclusterkey@0"
I20260811 12:49:35.601086 28962 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 12:49:35.618072 29006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:35.619664 28962 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:49:35.619853 28962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "192e00576a964e7aa82d94d918b9e5ef"
format_stamp: "Formatted at 2026-08-11 12:49:35 on dist-test-slave-f0tn"
server_key: "39897227bca9c6fd025515936c895403"
server_key_iv: "6e88d52cd5ad902a8efbc9458b250a84"
server_key_version: "kuduclusterkey@0"
I20260811 12:49:35.620077 28962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:35.648247 28962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:35.652201 28962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:35.652484 28962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:35.653517 28962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:49:35.655102 28962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:49:35.655180 28962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:35.655299 28962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:49:35.655351 28962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:35.690397 28962 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:41945
I20260811 12:49:35.690416 29214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:41945 every 8 connection(s)
I20260811 12:49:35.691852 28962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:49:35.701967 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 28962
I20260811 12:49:35.702142 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:49:35.702567 7613 external_mini_cluster.cc:1428] Setting key 13a3580d9683ecd7287f3fb946a37e29
Aug 11 12:49:35 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452575, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:49:35.714313 29219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:35.694544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:41053 (local address 127.7.111.126:36943)
0811 12:49:35.695035 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:35.695041 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:35.696106 (+ 1065us) server_negotiation.cc:407] Connection header received
0811 12:49:35.698673 (+ 2567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:35.698677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:35.698759 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:35.698854 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:35.700759 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:35.701550 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:35.703885 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:35.704394 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:35.704502 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:35.707823 (+ 3321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:35.707829 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:35.707833 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:35.707871 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:35.710995 (+ 3124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:35.711977 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:35.711983 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:35.711987 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:35.712138 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:35.712564 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:35.712575 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:35.712578 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:35.712922 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:35.713053 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:35.713760 (+ 707us) server_negotiation.cc:299] Negotiation successful
0811 12:49:35.713974 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":232,"threads_started":1}
I20260811 12:49:35.715256 29220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:35.695046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:36943 (local address 127.7.111.66:41053)
0811 12:49:35.695909 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:35.695959 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:35.698382 (+ 2423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:35.699135 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:35.699145 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:35.699756 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:49:35.700555 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:35.700587 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:35.701771 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:35.701777 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:35.703749 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:35.703763 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:35.704624 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:35.704631 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:35.704839 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:35.705538 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:49:35.705561 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:49:35.707594 (+ 2033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:49:35.711184 (+ 3590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:35.711193 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:35.711211 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:35.711846 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:35.712265 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:35.712269 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:35.712271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:35.712463 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:35.713031 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:49:35.713036 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:49:35.713344 (+ 308us) client_negotiation.cc:769] Sending connection context
0811 12:49:35.713656 (+ 312us) client_negotiation.cc:240] Negotiation successful
0811 12:49:35.714034 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":314,"threads_started":1}
I20260811 12:49:35.716696 29215 heartbeater.cc:347] Connected to a master server at 127.7.111.126:36943
I20260811 12:49:35.716979 29215 heartbeater.cc:464] Registering TS with master...
I20260811 12:49:35.717696 29215 heartbeater.cc:511] Master 127.7.111.126:36943 requested a full tablet report, sending...
I20260811 12:49:35.719097 28563 ts_manager.cc:194] Registered new tserver with Master: 192e00576a964e7aa82d94d918b9e5ef (127.7.111.66:41945)
I20260811 12:49:35.719882 28563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:41053
WARNING: no policy specified for kudu/127.7.111.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
I20260811 12:49:35.766700 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.111.84:46741/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:36943
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:41997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:49:35.952608 28955 heartbeater.cc:503] Master 127.7.111.126:36943 was elected leader, sending a full tablet report...
W20260811 12:49:36.102294 29227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:36.102638 29227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:36.102792 29227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:49:36.110066 29227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:36.110165 29227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:36.110316 29227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:49:36.119956 29227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:41997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.111.84:46741/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:36943
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:36.121215 29227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:36.122596 29227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:36.135291 29234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:36.135368 29235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:36.135330 29240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:36.138790 29227 server_base.cc:1034] running on GCE node
I20260811 12:49:36.139346 29227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:36.140256 29227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:36.141489 29227 hybrid_clock.cc:648] HybridClock initialized: now 1786452576141422 us; error 53 us; skew 500 ppm
I20260811 12:49:36.141832 29227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:36 dist-test-slave-f0tn krb5kdc[27751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452576, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:36.144897 29227 init.cc:377] Logged in from keytab as kudu/127.7.111.67@KRBTEST.COM (short username kudu)
I20260811 12:49:36.146440 29227 webserver.cc:460] Webserver started at http://127.7.111.67:43461/ using document root <none> and password file <none>
I20260811 12:49:36.147161 29227 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:36.147275 29227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:49:36.147580 29227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:49:36 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452576, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
I20260811 12:49:36.164108 29227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "bff68f2564244bab97d628c2ecc3c156"
format_stamp: "Formatted at 2026-08-11 12:49:36 on dist-test-slave-f0tn"
server_key: "75eb84f1ae78908b590c567512b197a1"
server_key_iv: "1bf22fcfd2837ed2e89d866e8752ff07"
server_key_version: "kuduclusterkey@0"
I20260811 12:49:36.164958 29227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "bff68f2564244bab97d628c2ecc3c156"
format_stamp: "Formatted at 2026-08-11 12:49:36 on dist-test-slave-f0tn"
server_key: "75eb84f1ae78908b590c567512b197a1"
server_key_iv: "1bf22fcfd2837ed2e89d866e8752ff07"
server_key_version: "kuduclusterkey@0"
I20260811 12:49:36.170667 29227 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 12:49:36.188134 29247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:36.189689 29227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:49:36.189878 29227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "bff68f2564244bab97d628c2ecc3c156"
format_stamp: "Formatted at 2026-08-11 12:49:36 on dist-test-slave-f0tn"
server_key: "75eb84f1ae78908b590c567512b197a1"
server_key_iv: "1bf22fcfd2837ed2e89d866e8752ff07"
server_key_version: "kuduclusterkey@0"
I20260811 12:49:36.190084 29227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:36.217667 29227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:36.221727 29227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:36.222031 29227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:36.222985 29227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:49:36.224437 29227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:49:36.224519 29227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:36.224615 29227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:49:36.224658 29227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:36.255393 29227 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:33647
I20260811 12:49:36.255429 29424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:33647 every 8 connection(s)
I20260811 12:49:36.256716 29227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:49:36.264514 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 29227
I20260811 12:49:36.264703 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:49:36.265196 7613 external_mini_cluster.cc:1428] Setting key 5fc1aedb8452baa173267c5f389bbd8b
Aug 11 12:49:36 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452576, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:49:36.276357 29427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:36.259269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:40187 (local address 127.7.111.126:36943)
0811 12:49:36.259791 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:36.259797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:36.260715 (+ 918us) server_negotiation.cc:407] Connection header received
0811 12:49:36.262524 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:36.262529 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:36.262620 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:36.262722 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:36.264082 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:36.264853 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:36.266749 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:36.267275 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:36.267381 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:36.270807 (+ 3426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:36.270813 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:36.270816 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:36.270851 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:36.273305 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:36.274054 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:36.274058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:36.274060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:36.274186 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:36.274619 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:36.274623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:36.274625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:36.274908 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:36.275038 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:36.275772 (+ 734us) server_negotiation.cc:299] Negotiation successful
0811 12:49:36.276000 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":231,"threads_started":1}
I20260811 12:49:36.277302 29428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:36.259840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:36943 (local address 127.7.111.67:40187)
0811 12:49:36.260549 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:36.260589 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:36.262274 (+ 1685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:36.262914 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:36.262922 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:36.263354 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:49:36.263945 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:36.263962 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:36.265026 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:36.265032 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:36.266602 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:36.266620 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:36.267482 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:36.267487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:36.267700 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:36.268366 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:49:36.268397 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:49:36.270573 (+ 2176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:49:36.273464 (+ 2891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:36.273473 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:36.273492 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:36.273929 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:36.274292 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:36.274297 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:36.274302 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:36.274505 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:36.275036 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:49:36.275044 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:49:36.275390 (+ 346us) client_negotiation.cc:769] Sending connection context
0811 12:49:36.275635 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 12:49:36.276176 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":262,"threads_started":1}
I20260811 12:49:36.278731 29425 heartbeater.cc:347] Connected to a master server at 127.7.111.126:36943
I20260811 12:49:36.279057 29425 heartbeater.cc:464] Registering TS with master...
I20260811 12:49:36.279904 29425 heartbeater.cc:511] Master 127.7.111.126:36943 requested a full tablet report, sending...
I20260811 12:49:36.281347 28563 ts_manager.cc:194] Registered new tserver with Master: bff68f2564244bab97d628c2ecc3c156 (127.7.111.67:33647)
I20260811 12:49:36.282181 28563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:40187
I20260811 12:49:36.292759 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:49:36.294477 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 28705
Aug 11 12:49:36 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.65@KRBTEST.COM
I20260811 12:49:36.310488 29432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:36.295436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55050 (local address 127.7.111.65:40037)
0811 12:49:36.296051 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:36.296058 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:36.296083 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:49:36.296145 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:36.296151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:36.296351 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:36.296482 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:36.297926 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:36.298955 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:36.300395 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:36.300958 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:36.301056 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:36.305196 (+ 4140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:36.305210 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:36.305227 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:36.305266 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:36.307958 (+ 2692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:36.308587 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:36.308595 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:36.308597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:36.308758 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:36.309114 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:36.309117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:36.309119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:36.309521 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:36.309776 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:36.309989 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 12:49:36.310203 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":308,"threads_started":1}
I20260811 12:49:36.311885 28947 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:49:36.394474 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 28705
I20260811 12:49:36.419279 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 28962
Aug 11 12:49:36 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.66@KRBTEST.COM
I20260811 12:49:36.434742 29435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:36.420210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51754 (local address 127.7.111.66:41945)
0811 12:49:36.420726 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:36.420734 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:36.420761 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:49:36.420835 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:36.420842 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:36.421050 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:36.421199 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:36.422418 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:36.423446 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:36.424744 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:36.425491 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:36.425595 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:36.429665 (+ 4070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:36.429675 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:36.429689 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:36.429730 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:36.432184 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:36.432765 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:36.432771 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:36.432773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:36.432895 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:36.433226 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:36.433230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:36.433233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:36.433659 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:36.433901 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:36.434253 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 12:49:36.434454 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":271,"threads_started":1}
I20260811 12:49:36.435973 29205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:49:36.518185 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 28962
I20260811 12:49:36.538188 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 29227
Aug 11 12:49:36 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.67@KRBTEST.COM
I20260811 12:49:36.553893 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:36.539127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45958 (local address 127.7.111.67:33647)
0811 12:49:36.539814 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:36.539821 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:36.539850 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:49:36.539911 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:36.539917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:36.540153 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:36.540293 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:36.541769 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:36.542815 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:36.544041 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:36.544573 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:36.544665 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:36.548947 (+ 4282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:36.548959 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:36.548974 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:36.549011 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:36.551538 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:36.552122 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:36.552129 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:36.552131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:36.552264 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:36.552518 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:36.552521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:36.552523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:36.552841 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:36.553070 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:36.553305 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 12:49:36.553536 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":323,"threads_started":1}
I20260811 12:49:36.555135 29419 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:49:36.637321 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 29227
I20260811 12:49:36.660818 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 28510
I20260811 12:49:36.662156 28615 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:49:36.768929 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 28510
I20260811 12:49:36.805871 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:36943
--webserver_interface=127.7.111.126
--webserver_port=41441
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:41997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:36943
--ranger_config_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.111.84:46741/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.111.84:46741/kms with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:49:37.144217 29447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:37.144506 29447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:37.144646 29447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:49:37.151932 29447 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:49:37.152062 29447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:37.152149 29447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:37.152220 29447 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:49:37.152275 29447 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:49:37.163488 29447 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:41997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.111.84:46741/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:36943
--ranger_config_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:36943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=41441
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:37.164737 29447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:37.166172 29447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:37.178035 29458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:37.177933 29459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:37.177969 29463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:37.178846 29447 server_base.cc:1034] running on GCE node
I20260811 12:49:37.179636 29447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:37.181015 29447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:37.182264 29447 hybrid_clock.cc:648] HybridClock initialized: now 1786452577182202 us; error 48 us; skew 500 ppm
I20260811 12:49:37.182595 29447 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:37 dist-test-slave-f0tn krb5kdc[27751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452577, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:37.185477 29447 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:49:37.186862 29447 webserver.cc:460] Webserver started at http://127.7.111.126:41441/ using document root <none> and password file <none>
I20260811 12:49:37.187479 29447 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:37.187598 29447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 12:49:37 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452577, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
I20260811 12:49:37.777313 29447 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 12:49:37.780436 29474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:37.782064 29447 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:49:37.782230 29447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "c42b56beae984aeab6fda7f7b76d8d96"
format_stamp: "Formatted at 2026-08-11 12:49:33 on dist-test-slave-f0tn"
server_key: "f3d08d23d0043107e6ff6d954af8e621"
server_key_iv: "02ad7a2be42dd1dd9d01ef78376008ee"
server_key_version: "kuduclusterkey@0"
I20260811 12:49:37.783212 29447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:37.796627 29447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:37.800458 29447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:37.800726 29447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:37.819175 29447 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:36943
I20260811 12:49:37.819195 29559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:36943 every 8 connection(s)
I20260811 12:49:37.820775 29447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:49:37.827646 29560 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:49:37.830845 29560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96: Bootstrap starting.
I20260811 12:49:37.831035 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 29447
I20260811 12:49:37.831861 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.111.84:46741/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:40037
--local_ip_for_outbound_sockets=127.7.111.65
--tserver_master_addrs=127.7.111.126:36943
--webserver_port=42857
--webserver_interface=127.7.111.65
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:41997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.111.84:46741/kms with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:49:37.849170 29560 log.cc:824] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96: Log is configured to *not* fsync() on all Append() calls
I20260811 12:49:37.857822 29560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:49:37.858301 29560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96: Bootstrap complete.
I20260811 12:49:37.867080 29560 raft_consensus.cc:348] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42b56beae984aeab6fda7f7b76d8d96" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36943 } }
I20260811 12:49:37.867928 29560 raft_consensus.cc:729] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c42b56beae984aeab6fda7f7b76d8d96, State: Initialized, Role: FOLLOWER
I20260811 12:49:37.868521 29560 consensus_queue.cc:260] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42b56beae984aeab6fda7f7b76d8d96" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36943 } }
I20260811 12:49:37.868728 29560 raft_consensus.cc:388] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:49:37.868835 29560 raft_consensus.cc:482] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:49:37.868983 29560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [term 1 FOLLOWER]: Advancing to term 2
I20260811 12:49:37.872650 29560 raft_consensus.cc:504] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42b56beae984aeab6fda7f7b76d8d96" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36943 } }
I20260811 12:49:37.873095 29560 leader_election.cc:302] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c42b56beae984aeab6fda7f7b76d8d96; no voters:
I20260811 12:49:37.873675 29560 leader_election.cc:288] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 12:49:37.873842 29569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 12:49:37.875665 29560 sys_catalog.cc:564] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:49:37.877635 29569 raft_consensus.cc:686] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [term 2 LEADER]: Becoming Leader. State: Replica: c42b56beae984aeab6fda7f7b76d8d96, State: Running, Role: LEADER
I20260811 12:49:37.878321 29569 consensus_queue.cc:237] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42b56beae984aeab6fda7f7b76d8d96" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36943 } }
I20260811 12:49:37.881407 29570 sys_catalog.cc:455] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c42b56beae984aeab6fda7f7b76d8d96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42b56beae984aeab6fda7f7b76d8d96" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36943 } } }
I20260811 12:49:37.881685 29570 sys_catalog.cc:458] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [sys.catalog]: This master's current role is: LEADER
I20260811 12:49:37.880790 29571 sys_catalog.cc:455] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c42b56beae984aeab6fda7f7b76d8d96. Latest consensus state: current_term: 2 leader_uuid: "c42b56beae984aeab6fda7f7b76d8d96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42b56beae984aeab6fda7f7b76d8d96" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 36943 } } }
I20260811 12:49:37.882262 29571 sys_catalog.cc:458] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96 [sys.catalog]: This master's current role is: LEADER
I20260811 12:49:37.893293 29585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:49:37.899436 29585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:49:37.900189 29560 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 12:49:37.900978 29585 catalog_manager.cc:1259] Loaded cluster ID: e6a521c5f42c4f6d8e4337bc82819bc6
I20260811 12:49:37.901140 29585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:49:37.905233 29585 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:49:37.906708 29585 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96: Loaded TSK: 0
I20260811 12:49:37.908303 29585 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 12:49:38.284487 29566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:38.284893 29566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:38.285053 29566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:49:38.294309 29566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:38.294456 29566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:38.294637 29566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:49:38.309470 29566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:41997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.111.84:46741/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:40037
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=42857
--tserver_master_addrs=127.7.111.126:36943
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:38.311275 29566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:38.313156 29566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:38.331348 29623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:38.335474 29626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:38.343798 29629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:38.344051 29566 server_base.cc:1034] running on GCE node
I20260811 12:49:38.345117 29566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:38.348361 29566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:38.349642 29566 hybrid_clock.cc:648] HybridClock initialized: now 1786452578349551 us; error 71 us; skew 500 ppm
I20260811 12:49:38.350103 29566 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:38 dist-test-slave-f0tn krb5kdc[27751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452578, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:38.357868 29566 init.cc:377] Logged in from keytab as kudu/127.7.111.65@KRBTEST.COM (short username kudu)
I20260811 12:49:38.360047 29566 webserver.cc:460] Webserver started at http://127.7.111.65:42857/ using document root <none> and password file <none>
I20260811 12:49:38.361088 29566 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:38.361253 29566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 12:49:38 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452578, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
I20260811 12:49:38.417621 29566 fs_manager.cc:709] Time spent opening directory manager: real 0.019s user 0.013s sys 0.004s
I20260811 12:49:38.423211 29646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:38.425271 29566 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.006s sys 0.000s
I20260811 12:49:38.425580 29566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "25f13b7984a7401ab3623c90a9efa2b0"
format_stamp: "Formatted at 2026-08-11 12:49:34 on dist-test-slave-f0tn"
server_key: "5247c030a1565f7375605328e827a668"
server_key_iv: "880e64e202835566dff37cd8a80a8451"
server_key_version: "kuduclusterkey@0"
I20260811 12:49:38.427204 29566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:38.462961 29566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:38.467516 29566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:38.467820 29566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:38.468948 29566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:49:38.470572 29566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:49:38.470692 29566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:38.470810 29566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:49:38.470896 29566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:38.538914 29566 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:40037
I20260811 12:49:38.538980 29853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:40037 every 8 connection(s)
I20260811 12:49:38.540596 29566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:49:38.546196 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 29566
I20260811 12:49:38.547184 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.111.84:46741/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:41945
--local_ip_for_outbound_sockets=127.7.111.66
--tserver_master_addrs=127.7.111.126:36943
--webserver_port=34053
--webserver_interface=127.7.111.66
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:41997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.111.84:46741/kms with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:49:38 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452578, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:49:38.587404 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:38.550278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:36943 (local address 127.7.111.65:38549)
0811 12:49:38.551327 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:38.551396 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:38.554082 (+ 2686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:38.567621 (+ 13539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:38.567632 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:38.568246 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:49:38.568859 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:38.568886 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:38.571340 (+ 2454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:38.571346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:38.572907 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:38.572920 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:38.574159 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:38.574169 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:38.574424 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:38.575253 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:49:38.575277 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:49:38.577482 (+ 2205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:49:38.581850 (+ 4368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:38.581860 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:38.581876 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:38.582469 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:38.583033 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:38.583038 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:38.583041 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:38.583318 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:38.584457 (+ 1139us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:49:38.584464 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:49:38.584871 (+ 407us) client_negotiation.cc:769] Sending connection context
0811 12:49:38.585208 (+ 337us) client_negotiation.cc:240] Negotiation successful
0811 12:49:38.585807 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"thread_start_us":411,"threads_started":1}
I20260811 12:49:38.588522 29856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:38.545026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:38549 (local address 127.7.111.126:36943)
0811 12:49:38.565999 (+ 20973us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:38.566020 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:38.566076 (+ 56us) server_negotiation.cc:407] Connection header received
0811 12:49:38.566459 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:38.566476 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:38.566974 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:38.567444 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:38.569129 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:38.571123 (+ 1994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:38.573071 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:38.573908 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:38.574087 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:38.577742 (+ 3655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:38.577750 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:38.577767 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:38.577810 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:38.581660 (+ 3850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:38.582648 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:38.582658 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:38.582666 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:38.582885 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:38.583471 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:38.583477 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:38.583480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:38.584169 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:38.584361 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:38.586509 (+ 2148us) server_negotiation.cc:299] Negotiation successful
0811 12:49:38.586944 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20679,"thread_start_us":20427,"threads_started":1}
I20260811 12:49:38.592643 29854 heartbeater.cc:347] Connected to a master server at 127.7.111.126:36943
I20260811 12:49:38.593000 29854 heartbeater.cc:464] Registering TS with master...
I20260811 12:49:38.594035 29854 heartbeater.cc:511] Master 127.7.111.126:36943 requested a full tablet report, sending...
W20260811 12:49:38.597712 29854 heartbeater.cc:644] Failed to heartbeat to 127.7.111.126:36943 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 12:49:38.597954 29854 heartbeater.cc:464] Registering TS with master...
I20260811 12:49:38.598246 29854 heartbeater.cc:511] Master 127.7.111.126:36943 requested a full tablet report, sending...
I20260811 12:49:38.599485 29854 heartbeater.cc:464] Registering TS with master...
I20260811 12:49:38.599762 29854 heartbeater.cc:511] Master 127.7.111.126:36943 requested a full tablet report, sending...
W20260811 12:49:38.600889 29854 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 12:49:38.958060 29860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:38.958369 29860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:38.958506 29860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:49:38.968250 29860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:38.968390 29860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:38.968592 29860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:49:38.981649 29860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:41997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.111.84:46741/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:41945
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=34053
--tserver_master_addrs=127.7.111.126:36943
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:38.983397 29860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:38.985167 29860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:39.000933 29888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:39.008653 29894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:39.010001 29898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:39.010669 29860 server_base.cc:1034] running on GCE node
I20260811 12:49:39.011361 29860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:39.012487 29860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:39.013774 29860 hybrid_clock.cc:648] HybridClock initialized: now 1786452579013726 us; error 35 us; skew 500 ppm
I20260811 12:49:39.014110 29860 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:39 dist-test-slave-f0tn krb5kdc[27751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452579, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:39.018102 29860 init.cc:377] Logged in from keytab as kudu/127.7.111.66@KRBTEST.COM (short username kudu)
I20260811 12:49:39.020769 29860 webserver.cc:460] Webserver started at http://127.7.111.66:34053/ using document root <none> and password file <none>
I20260811 12:49:39.021834 29860 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:39.021999 29860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 12:49:39 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452579, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
I20260811 12:49:39.079200 29860 fs_manager.cc:709] Time spent opening directory manager: real 0.014s user 0.008s sys 0.004s
I20260811 12:49:39.094208 29912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:39.096764 29860 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.004s sys 0.008s
I20260811 12:49:39.096997 29860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "192e00576a964e7aa82d94d918b9e5ef"
format_stamp: "Formatted at 2026-08-11 12:49:35 on dist-test-slave-f0tn"
server_key: "39897227bca9c6fd025515936c895403"
server_key_iv: "6e88d52cd5ad902a8efbc9458b250a84"
server_key_version: "kuduclusterkey@0"
I20260811 12:49:39.098546 29860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:39.119534 29860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:39.125137 29860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:39.125495 29860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:39.129868 29860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:49:39.132936 29860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:49:39.133116 29860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:39.133271 29860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:49:39.133397 29860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:49:39 dist-test-slave-f0tn krb5kdc[27751](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786452579, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:39.200014 29860 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:41945
I20260811 12:49:39.200079 30109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:41945 every 8 connection(s)
I20260811 12:49:39.201865 29860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:49:39.209754 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 29860
I20260811 12:49:39.210615 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.111.84:46741/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:33647
--local_ip_for_outbound_sockets=127.7.111.67
--tserver_master_addrs=127.7.111.126:36943
--webserver_port=43461
--webserver_interface=127.7.111.67
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:41997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.111.84:46741/kms with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:49:39 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452579, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:49:39.226528 30112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:39.204810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:35671 (local address 127.7.111.126:36943)
0811 12:49:39.205269 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:39.205273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:39.206658 (+ 1385us) server_negotiation.cc:407] Connection header received
0811 12:49:39.208717 (+ 2059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:39.208724 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:39.208820 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:39.208921 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:39.210543 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:39.211434 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:39.214002 (+ 2568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:39.214693 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:39.214818 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:39.218827 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:39.218834 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:39.218839 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:39.218884 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:39.222650 (+ 3766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:39.223653 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:39.223659 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:39.223664 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:39.223826 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:39.224467 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:39.224473 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:39.224476 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:39.224874 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:39.225037 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:39.226061 (+ 1024us) server_negotiation.cc:299] Negotiation successful
0811 12:49:39.226263 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":236,"threads_started":1}
I20260811 12:49:39.228233 30113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:39.205595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:36943 (local address 127.7.111.66:35671)
0811 12:49:39.206473 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:39.206521 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:39.208405 (+ 1884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:39.209192 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:39.209201 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:39.209704 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:49:39.210358 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:39.210377 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:39.211621 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:39.211627 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:39.213787 (+ 2160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:39.213806 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:39.215131 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:39.215136 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:39.215360 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:39.216081 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:49:39.216109 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:49:39.218512 (+ 2403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:49:39.222857 (+ 4345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:39.222868 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:39.222890 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:39.223497 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:39.223975 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:39.223981 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:39.223985 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:39.224310 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:39.225034 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:49:39.225044 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:49:39.225593 (+ 549us) client_negotiation.cc:769] Sending connection context
0811 12:49:39.225939 (+ 346us) client_negotiation.cc:240] Negotiation successful
0811 12:49:39.226615 (+ 676us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":338,"threads_started":1}
I20260811 12:49:39.229997 30110 heartbeater.cc:347] Connected to a master server at 127.7.111.126:36943
I20260811 12:49:39.230379 30110 heartbeater.cc:464] Registering TS with master...
I20260811 12:49:39.231292 30110 heartbeater.cc:511] Master 127.7.111.126:36943 requested a full tablet report, sending...
I20260811 12:49:39.233494 29502 ts_manager.cc:194] Registered new tserver with Master: 192e00576a964e7aa82d94d918b9e5ef (127.7.111.66:41945)
I20260811 12:49:39.235627 29502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:35671
I20260811 12:49:39.601064 29854 heartbeater.cc:464] Registering TS with master...
I20260811 12:49:39.601557 29854 heartbeater.cc:511] Master 127.7.111.126:36943 requested a full tablet report, sending...
Aug 11 12:49:39 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452579, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
I20260811 12:49:39.602882 29502 ts_manager.cc:194] Registered new tserver with Master: 25f13b7984a7401ab3623c90a9efa2b0 (127.7.111.65:40037)
I20260811 12:49:39.604076 29502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:38549
W20260811 12:49:39.675818 30117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:39.676265 30117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:39.676491 30117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:49:39.687711 30117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:39.687846 30117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:39.688050 30117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:49:39.702527 30117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:41997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.111.84:46741/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:33647
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=43461
--tserver_master_addrs=127.7.111.126:36943
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:39.703940 30117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:39.705502 30117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:39.721069 30129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:39.721087 30126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:39.727494 30117 server_base.cc:1034] running on GCE node
W20260811 12:49:39.727803 30134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:39.728483 30117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:39.729697 30117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:39.730964 30117 hybrid_clock.cc:648] HybridClock initialized: now 1786452579730911 us; error 50 us; skew 500 ppm
I20260811 12:49:39.731302 30117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:39 dist-test-slave-f0tn krb5kdc[27751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452579, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:39.734941 30117 init.cc:377] Logged in from keytab as kudu/127.7.111.67@KRBTEST.COM (short username kudu)
I20260811 12:49:39.736707 30117 webserver.cc:460] Webserver started at http://127.7.111.67:43461/ using document root <none> and password file <none>
I20260811 12:49:39.737478 30117 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:39.737615 30117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 12:49:39 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452579, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
I20260811 12:49:39.764221 30117 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 12:49:39.767084 30150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:39.768484 30117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:49:39.768652 30117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "bff68f2564244bab97d628c2ecc3c156"
format_stamp: "Formatted at 2026-08-11 12:49:36 on dist-test-slave-f0tn"
server_key: "75eb84f1ae78908b590c567512b197a1"
server_key_iv: "1bf22fcfd2837ed2e89d866e8752ff07"
server_key_version: "kuduclusterkey@0"
I20260811 12:49:39.769737 30117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:39.791193 30117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:39.795401 30117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:39.795679 30117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:39.796767 30117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:49:39.798272 30117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:49:39.798368 30117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:39.798491 30117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:49:39.798565 30117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:49:39 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452579, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for HTTP/127.7.111.84@KRBTEST.COM
I20260811 12:49:39.894783 30117 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:33647
I20260811 12:49:39.894867 30332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:33647 every 8 connection(s)
I20260811 12:49:39.896946 30117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:49:39.908443 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 30117
Aug 11 12:49:39 dist-test-slave-f0tn krb5kdc[27751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452579, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:49:39.933679 30341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:39.900706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:36943 (local address 127.7.111.67:49899)
0811 12:49:39.905348 (+ 4642us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:39.905440 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:39.908067 (+ 2627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:39.909605 (+ 1538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:39.909619 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:39.910238 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:49:39.911115 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:39.911141 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:39.912926 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:39.912936 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:39.915038 (+ 2102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:39.915068 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:39.916251 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:39.916258 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:39.916569 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:39.921593 (+ 5024us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:49:39.921627 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:49:39.924544 (+ 2917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:49:39.928759 (+ 4215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:39.928770 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:39.928790 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:39.929421 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:39.929920 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:39.929926 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:39.929929 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:39.930211 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:39.930911 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:49:39.930921 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:49:39.931343 (+ 422us) client_negotiation.cc:769] Sending connection context
0811 12:49:39.931641 (+ 298us) client_negotiation.cc:240] Negotiation successful
0811 12:49:39.932206 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4317,"thread_start_us":4087,"threads_started":1}
I20260811 12:49:39.935179 30342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:39.907694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:49899 (local address 127.7.111.126:36943)
0811 12:49:39.908791 (+ 1097us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:39.908797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:39.908818 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:49:39.908884 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:39.908890 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:39.909078 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:39.911345 (+ 2267us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:39.911554 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:39.912716 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:39.915256 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:39.915936 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:39.916952 (+ 1016us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:39.924871 (+ 7919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:39.924879 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:39.924884 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:39.924927 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:39.928585 (+ 3658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:39.929622 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:39.929626 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:39.929630 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:39.929777 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:39.930368 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:39.930373 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:39.930377 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:39.930745 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:39.934437 (+ 3692us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:39.934579 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 12:49:39.934814 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":850,"thread_start_us":2074,"threads_started":1}
I20260811 12:49:39.937992 30333 heartbeater.cc:347] Connected to a master server at 127.7.111.126:36943
I20260811 12:49:39.938397 30333 heartbeater.cc:464] Registering TS with master...
I20260811 12:49:39.939378 30333 heartbeater.cc:511] Master 127.7.111.126:36943 requested a full tablet report, sending...
I20260811 12:49:39.941092 29502 ts_manager.cc:194] Registered new tserver with Master: bff68f2564244bab97d628c2ecc3c156 (127.7.111.67:33647)
I20260811 12:49:39.942296 29502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:49899
I20260811 12:49:39.964272 30344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:39.919575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60130 (local address 127.7.111.126:36943)
0811 12:49:39.929842 (+ 10267us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:39.929849 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:39.945904 (+ 16055us) server_negotiation.cc:407] Connection header received
0811 12:49:39.946094 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:39.946099 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:39.946190 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:39.946299 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:39.948413 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:39.949651 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:39.951735 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:39.952396 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:39.952494 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:39.956662 (+ 4168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:39.956671 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:39.956677 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:39.956723 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:39.960455 (+ 3732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:39.961510 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:39.961517 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:39.961522 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:39.961696 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:39.962225 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:39.962231 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:39.962235 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:39.962645 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:39.962911 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:39.963575 (+ 664us) server_negotiation.cc:299] Negotiation successful
0811 12:49:39.963818 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10125,"thread_start_us":11726,"threads_started":1}
I20260811 12:49:39.968706 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:49:39.997227 30344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:39.976246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60134 (local address 127.7.111.126:36943)
0811 12:49:39.976529 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:39.976537 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:39.977954 (+ 1417us) server_negotiation.cc:407] Connection header received
0811 12:49:39.978293 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:39.978300 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:39.978393 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:39.978478 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:39.981043 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:39.982320 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:39.984542 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:39.985252 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:39.985416 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:39.989607 (+ 4191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:39.989615 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:39.989621 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:39.989669 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:39.993490 (+ 3821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:39.994455 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:39.994461 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:39.994465 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:39.994667 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:39.995215 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:39.995221 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:39.995225 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:39.995624 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:39.995860 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:39.996553 (+ 693us) server_negotiation.cc:299] Negotiation successful
0811 12:49:39.996818 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260811 12:49:40.004863 29502 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60134:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:49:40.014653 29502 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:49:40.084932 30358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:40.050439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:40037 (local address 127.0.0.1:55062)
0811 12:49:40.061498 (+ 11059us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:40.061550 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:40.061781 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:40.066585 (+ 4804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:40.066596 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:40.066634 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:49:40.067148 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:40.067163 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.070643 (+ 3480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.070649 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:40.073244 (+ 2595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:40.073262 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.083850 (+ 10588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.083857 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:40.084088 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:40.084158 (+ 70us) client_negotiation.cc:769] Sending connection context
0811 12:49:40.084308 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 12:49:40.084539 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10752,"thread_start_us":10628,"threads_started":1}
I20260811 12:49:40.087634 30364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:40.051596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55062 (local address 127.7.111.65:40037)
0811 12:49:40.065341 (+ 13745us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:40.065351 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:40.065407 (+ 56us) server_negotiation.cc:407] Connection header received
0811 12:49:40.065525 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:40.065535 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:40.065784 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:40.066012 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:49:40.068673 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.070433 (+ 1760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.082026 (+ 11593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.083604 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.083753 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:40.084043 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:40.087018 (+ 2975us) server_negotiation.cc:299] Negotiation successful
0811 12:49:40.087222 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13552,"thread_start_us":13446,"threads_started":1}
I20260811 12:49:40.088743 30360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:40.050838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:33647 (local address 127.0.0.1:45960)
0811 12:49:40.063077 (+ 12239us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:40.063098 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:40.063228 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:40.067871 (+ 4643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:40.067877 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:40.067908 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:49:40.068432 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:40.068445 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.075548 (+ 7103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.075554 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:40.079413 (+ 3859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:40.079431 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.088177 (+ 8746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.088183 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:40.088311 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:40.088337 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 12:49:40.088404 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 12:49:40.088501 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12096,"thread_start_us":15406,"threads_started":1}
I20260811 12:49:40.092103 29732 tablet_service.cc:1467] Processing CreateTablet for tablet 364c32e124544289a8eecea049ee08a0 (DEFAULT_TABLE table=test-table [id=9f9f44e3c72d402ea07e0419be61f5a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:49:40.092768 30363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:40.051050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45960 (local address 127.7.111.67:33647)
0811 12:49:40.063820 (+ 12770us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:40.063831 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:40.063877 (+ 46us) server_negotiation.cc:407] Connection header received
0811 12:49:40.064011 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:40.064021 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:40.064299 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:40.064526 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:49:40.073534 (+ 9008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.075331 (+ 1797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.084165 (+ 8834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.087287 (+ 3122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.087444 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:40.092091 (+ 4647us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:40.092179 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 12:49:40.092377 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12566,"thread_start_us":13783,"threads_started":1}
I20260811 12:49:40.093842 29732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 364c32e124544289a8eecea049ee08a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:49:40.094157 30365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:40.062063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:41945 (local address 127.0.0.1:51764)
0811 12:49:40.077166 (+ 15103us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:40.077186 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:40.077323 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:40.079605 (+ 2282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:40.079610 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:40.079640 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:49:40.080017 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:40.080030 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.085297 (+ 5267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.085302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:40.089825 (+ 4523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:40.089843 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.093195 (+ 3352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.093203 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:40.093468 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:40.093508 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 12:49:40.093610 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 12:49:40.093757 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14971,"thread_start_us":14774,"threads_started":1}
I20260811 12:49:40.094192 30366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:40.062313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51764 (local address 127.7.111.66:41945)
0811 12:49:40.067774 (+ 5461us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:40.067784 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:40.077502 (+ 9718us) server_negotiation.cc:407] Connection header received
0811 12:49:40.077631 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:40.077640 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:40.077923 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:40.078171 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:49:40.080249 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.081784 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.090067 (+ 8283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.091562 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.091729 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:40.091933 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:40.093711 (+ 1778us) server_negotiation.cc:299] Negotiation successful
0811 12:49:40.093894 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5274,"thread_start_us":5143,"threads_started":1}
I20260811 12:49:40.098238 29994 tablet_service.cc:1467] Processing CreateTablet for tablet 364c32e124544289a8eecea049ee08a0 (DEFAULT_TABLE table=test-table [id=9f9f44e3c72d402ea07e0419be61f5a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:49:40.098238 30223 tablet_service.cc:1467] Processing CreateTablet for tablet 364c32e124544289a8eecea049ee08a0 (DEFAULT_TABLE table=test-table [id=9f9f44e3c72d402ea07e0419be61f5a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:49:40.099684 30223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 364c32e124544289a8eecea049ee08a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:49:40.099684 29994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 364c32e124544289a8eecea049ee08a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:49:40.129094 30369 tablet_bootstrap.cc:492] T 364c32e124544289a8eecea049ee08a0 P 192e00576a964e7aa82d94d918b9e5ef: Bootstrap starting.
I20260811 12:49:40.135447 30369 tablet_bootstrap.cc:654] T 364c32e124544289a8eecea049ee08a0 P 192e00576a964e7aa82d94d918b9e5ef: Neither blocks nor log segments found. Creating new log.
I20260811 12:49:40.136263 30372 tablet_bootstrap.cc:492] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0: Bootstrap starting.
I20260811 12:49:40.142884 30372 tablet_bootstrap.cc:654] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0: Neither blocks nor log segments found. Creating new log.
I20260811 12:49:40.143940 30367 tablet_bootstrap.cc:492] T 364c32e124544289a8eecea049ee08a0 P bff68f2564244bab97d628c2ecc3c156: Bootstrap starting.
I20260811 12:49:40.145633 30369 log.cc:824] T 364c32e124544289a8eecea049ee08a0 P 192e00576a964e7aa82d94d918b9e5ef: Log is configured to *not* fsync() on all Append() calls
I20260811 12:49:40.145988 30372 log.cc:824] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0: Log is configured to *not* fsync() on all Append() calls
I20260811 12:49:40.149771 30367 tablet_bootstrap.cc:654] T 364c32e124544289a8eecea049ee08a0 P bff68f2564244bab97d628c2ecc3c156: Neither blocks nor log segments found. Creating new log.
I20260811 12:49:40.151072 30367 log.cc:824] T 364c32e124544289a8eecea049ee08a0 P bff68f2564244bab97d628c2ecc3c156: Log is configured to *not* fsync() on all Append() calls
I20260811 12:49:40.155988 30372 tablet_bootstrap.cc:492] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0: No bootstrap required, opened a new log
I20260811 12:49:40.156384 30372 ts_tablet_manager.cc:1397] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.005s
I20260811 12:49:40.165254 30367 tablet_bootstrap.cc:492] T 364c32e124544289a8eecea049ee08a0 P bff68f2564244bab97d628c2ecc3c156: No bootstrap required, opened a new log
I20260811 12:49:40.165802 30367 ts_tablet_manager.cc:1397] T 364c32e124544289a8eecea049ee08a0 P bff68f2564244bab97d628c2ecc3c156: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.006s
I20260811 12:49:40.167816 30369 tablet_bootstrap.cc:492] T 364c32e124544289a8eecea049ee08a0 P 192e00576a964e7aa82d94d918b9e5ef: No bootstrap required, opened a new log
I20260811 12:49:40.167652 30372 raft_consensus.cc:348] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25f13b7984a7401ab3623c90a9efa2b0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40037 } } peers { permanent_uuid: "bff68f2564244bab97d628c2ecc3c156" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 33647 } } peers { permanent_uuid: "192e00576a964e7aa82d94d918b9e5ef" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 41945 } }
I20260811 12:49:40.168150 30369 ts_tablet_manager.cc:1397] T 364c32e124544289a8eecea049ee08a0 P 192e00576a964e7aa82d94d918b9e5ef: Time spent bootstrapping tablet: real 0.039s user 0.003s sys 0.011s
I20260811 12:49:40.168337 30372 raft_consensus.cc:374] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:49:40.168499 30372 raft_consensus.cc:729] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25f13b7984a7401ab3623c90a9efa2b0, State: Initialized, Role: FOLLOWER
I20260811 12:49:40.169344 30372 consensus_queue.cc:260] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25f13b7984a7401ab3623c90a9efa2b0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40037 } } peers { permanent_uuid: "bff68f2564244bab97d628c2ecc3c156" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 33647 } } peers { permanent_uuid: "192e00576a964e7aa82d94d918b9e5ef" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 41945 } }
I20260811 12:49:40.177938 30372 ts_tablet_manager.cc:1428] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0: Time spent starting tablet: real 0.021s user 0.016s sys 0.003s
I20260811 12:49:40.178406 29854 heartbeater.cc:503] Master 127.7.111.126:36943 was elected leader, sending a full tablet report...
I20260811 12:49:40.190317 30367 raft_consensus.cc:348] T 364c32e124544289a8eecea049ee08a0 P bff68f2564244bab97d628c2ecc3c156 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25f13b7984a7401ab3623c90a9efa2b0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40037 } } peers { permanent_uuid: "bff68f2564244bab97d628c2ecc3c156" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 33647 } } peers { permanent_uuid: "192e00576a964e7aa82d94d918b9e5ef" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 41945 } }
I20260811 12:49:40.190842 30367 raft_consensus.cc:374] T 364c32e124544289a8eecea049ee08a0 P bff68f2564244bab97d628c2ecc3c156 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:49:40.190965 30367 raft_consensus.cc:729] T 364c32e124544289a8eecea049ee08a0 P bff68f2564244bab97d628c2ecc3c156 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bff68f2564244bab97d628c2ecc3c156, State: Initialized, Role: FOLLOWER
I20260811 12:49:40.191083 30369 raft_consensus.cc:348] T 364c32e124544289a8eecea049ee08a0 P 192e00576a964e7aa82d94d918b9e5ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25f13b7984a7401ab3623c90a9efa2b0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40037 } } peers { permanent_uuid: "bff68f2564244bab97d628c2ecc3c156" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 33647 } } peers { permanent_uuid: "192e00576a964e7aa82d94d918b9e5ef" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 41945 } }
I20260811 12:49:40.191478 30369 raft_consensus.cc:374] T 364c32e124544289a8eecea049ee08a0 P 192e00576a964e7aa82d94d918b9e5ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:49:40.191579 30369 raft_consensus.cc:729] T 364c32e124544289a8eecea049ee08a0 P 192e00576a964e7aa82d94d918b9e5ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 192e00576a964e7aa82d94d918b9e5ef, State: Initialized, Role: FOLLOWER
I20260811 12:49:40.191695 30367 consensus_queue.cc:260] T 364c32e124544289a8eecea049ee08a0 P bff68f2564244bab97d628c2ecc3c156 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25f13b7984a7401ab3623c90a9efa2b0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40037 } } peers { permanent_uuid: "bff68f2564244bab97d628c2ecc3c156" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 33647 } } peers { permanent_uuid: "192e00576a964e7aa82d94d918b9e5ef" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 41945 } }
I20260811 12:49:40.192142 30369 consensus_queue.cc:260] T 364c32e124544289a8eecea049ee08a0 P 192e00576a964e7aa82d94d918b9e5ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25f13b7984a7401ab3623c90a9efa2b0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40037 } } peers { permanent_uuid: "bff68f2564244bab97d628c2ecc3c156" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 33647 } } peers { permanent_uuid: "192e00576a964e7aa82d94d918b9e5ef" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 41945 } }
I20260811 12:49:40.192999 30333 heartbeater.cc:503] Master 127.7.111.126:36943 was elected leader, sending a full tablet report...
I20260811 12:49:40.193768 30367 ts_tablet_manager.cc:1428] T 364c32e124544289a8eecea049ee08a0 P bff68f2564244bab97d628c2ecc3c156: Time spent starting tablet: real 0.028s user 0.014s sys 0.000s
I20260811 12:49:40.197628 30110 heartbeater.cc:503] Master 127.7.111.126:36943 was elected leader, sending a full tablet report...
I20260811 12:49:40.198452 30369 ts_tablet_manager.cc:1428] T 364c32e124544289a8eecea049ee08a0 P 192e00576a964e7aa82d94d918b9e5ef: Time spent starting tablet: real 0.030s user 0.012s sys 0.004s
W20260811 12:49:40.208705 30111 tablet.cc:2369] T 364c32e124544289a8eecea049ee08a0 P 192e00576a964e7aa82d94d918b9e5ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:49:40.298734 30377 raft_consensus.cc:482] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:49:40.299135 30377 raft_consensus.cc:504] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25f13b7984a7401ab3623c90a9efa2b0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40037 } } peers { permanent_uuid: "bff68f2564244bab97d628c2ecc3c156" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 33647 } } peers { permanent_uuid: "192e00576a964e7aa82d94d918b9e5ef" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 41945 } }
I20260811 12:49:40.301512 30377 leader_election.cc:288] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bff68f2564244bab97d628c2ecc3c156 (127.7.111.67:33647), 192e00576a964e7aa82d94d918b9e5ef (127.7.111.66:41945)
W20260811 12:49:40.304081 29855 tablet.cc:2369] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:49:40.310074 30363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:40.301929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:54485 (local address 127.7.111.67:33647)
0811 12:49:40.302168 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:40.302175 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:40.303321 (+ 1146us) server_negotiation.cc:407] Connection header received
0811 12:49:40.303574 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:40.303580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:40.303661 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:40.303734 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:49:40.304668 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.305656 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.307792 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.309057 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.309241 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:40.309331 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:40.309697 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 12:49:40.309829 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 12:49:40.310096 30394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:40.302413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:33647 (local address 127.7.111.65:54485)
0811 12:49:40.303210 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:40.303227 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:40.303376 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:40.303954 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:40.303959 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:40.303992 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:49:40.304502 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:40.304517 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.305785 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.305789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:40.307586 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:40.307603 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.309294 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.309300 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:40.309525 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:40.309553 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 12:49:40.309638 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 12:49:40.309765 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":1112,"threads_started":1}
I20260811 12:49:40.310417 30366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:40.302021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:46933 (local address 127.7.111.66:41945)
0811 12:49:40.302257 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:40.302262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:40.303941 (+ 1679us) server_negotiation.cc:407] Connection header received
0811 12:49:40.304265 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:40.304270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:40.304350 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:40.304414 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:49:40.305248 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.306379 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.308405 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.309223 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.309412 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:40.309486 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:40.309975 (+ 489us) server_negotiation.cc:299] Negotiation successful
0811 12:49:40.310131 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:49:40.311188 30393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:40.302449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:41945 (local address 127.7.111.65:46933)
0811 12:49:40.303848 (+ 1399us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:40.303865 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:40.304031 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:40.304649 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:40.304653 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:40.304690 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:49:40.305048 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:40.305059 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.306551 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.306557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:40.308253 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:40.308266 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.309542 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.309550 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:40.309740 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:40.309780 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 12:49:40.310618 (+ 838us) client_negotiation.cc:240] Negotiation successful
0811 12:49:40.310774 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1280,"spinlock_wait_cycles":22400,"thread_start_us":429,"threads_started":1}
I20260811 12:49:40.311299 30255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364c32e124544289a8eecea049ee08a0" candidate_uuid: "25f13b7984a7401ab3623c90a9efa2b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff68f2564244bab97d628c2ecc3c156" is_pre_election: true
I20260811 12:49:40.311872 30255 raft_consensus.cc:2454] T 364c32e124544289a8eecea049ee08a0 P bff68f2564244bab97d628c2ecc3c156 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25f13b7984a7401ab3623c90a9efa2b0 in term 0.
I20260811 12:49:40.312523 29649 leader_election.cc:302] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25f13b7984a7401ab3623c90a9efa2b0, bff68f2564244bab97d628c2ecc3c156; no voters:
I20260811 12:49:40.312480 30032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364c32e124544289a8eecea049ee08a0" candidate_uuid: "25f13b7984a7401ab3623c90a9efa2b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "192e00576a964e7aa82d94d918b9e5ef" is_pre_election: true
I20260811 12:49:40.312968 30377 raft_consensus.cc:2781] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:49:40.313038 30032 raft_consensus.cc:2454] T 364c32e124544289a8eecea049ee08a0 P 192e00576a964e7aa82d94d918b9e5ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25f13b7984a7401ab3623c90a9efa2b0 in term 0.
I20260811 12:49:40.313139 30377 raft_consensus.cc:482] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:49:40.313248 30377 raft_consensus.cc:3037] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:49:40.317015 30377 raft_consensus.cc:504] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25f13b7984a7401ab3623c90a9efa2b0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40037 } } peers { permanent_uuid: "bff68f2564244bab97d628c2ecc3c156" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 33647 } } peers { permanent_uuid: "192e00576a964e7aa82d94d918b9e5ef" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 41945 } }
I20260811 12:49:40.317873 30377 leader_election.cc:288] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0 [CANDIDATE]: Term 1 election: Requested vote from peers bff68f2564244bab97d628c2ecc3c156 (127.7.111.67:33647), 192e00576a964e7aa82d94d918b9e5ef (127.7.111.66:41945)
I20260811 12:49:40.318306 30255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364c32e124544289a8eecea049ee08a0" candidate_uuid: "25f13b7984a7401ab3623c90a9efa2b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff68f2564244bab97d628c2ecc3c156"
I20260811 12:49:40.318372 30032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364c32e124544289a8eecea049ee08a0" candidate_uuid: "25f13b7984a7401ab3623c90a9efa2b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "192e00576a964e7aa82d94d918b9e5ef"
I20260811 12:49:40.318552 30255 raft_consensus.cc:3037] T 364c32e124544289a8eecea049ee08a0 P bff68f2564244bab97d628c2ecc3c156 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:49:40.318562 30032 raft_consensus.cc:3037] T 364c32e124544289a8eecea049ee08a0 P 192e00576a964e7aa82d94d918b9e5ef [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:49:40.320782 30032 raft_consensus.cc:2454] T 364c32e124544289a8eecea049ee08a0 P 192e00576a964e7aa82d94d918b9e5ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25f13b7984a7401ab3623c90a9efa2b0 in term 1.
I20260811 12:49:40.321172 29647 leader_election.cc:302] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 192e00576a964e7aa82d94d918b9e5ef, 25f13b7984a7401ab3623c90a9efa2b0; no voters:
I20260811 12:49:40.321460 30377 raft_consensus.cc:2781] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:49:40.321997 30377 raft_consensus.cc:686] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0 [term 1 LEADER]: Becoming Leader. State: Replica: 25f13b7984a7401ab3623c90a9efa2b0, State: Running, Role: LEADER
I20260811 12:49:40.322027 30255 raft_consensus.cc:2454] T 364c32e124544289a8eecea049ee08a0 P bff68f2564244bab97d628c2ecc3c156 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25f13b7984a7401ab3623c90a9efa2b0 in term 1.
I20260811 12:49:40.322921 30377 consensus_queue.cc:237] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25f13b7984a7401ab3623c90a9efa2b0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40037 } } peers { permanent_uuid: "bff68f2564244bab97d628c2ecc3c156" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 33647 } } peers { permanent_uuid: "192e00576a964e7aa82d94d918b9e5ef" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 41945 } }
I20260811 12:49:40.327968 29501 catalog_manager.cc:5495] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 25f13b7984a7401ab3623c90a9efa2b0 (127.7.111.65). New cstate: current_term: 1 leader_uuid: "25f13b7984a7401ab3623c90a9efa2b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25f13b7984a7401ab3623c90a9efa2b0" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 40037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bff68f2564244bab97d628c2ecc3c156" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 33647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "192e00576a964e7aa82d94d918b9e5ef" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 41945 } health_report { overall_health: UNKNOWN } } }
I20260811 12:49:40.388012 30364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:40.382462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55070 (local address 127.7.111.65:40037)
0811 12:49:40.382720 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:40.382725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:40.382748 (+ 23us) server_negotiation.cc:407] Connection header received
0811 12:49:40.382964 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:40.382968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:40.383082 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:40.383219 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:49:40.383632 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.384386 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.385808 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.386197 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.386310 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:40.386852 (+ 542us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:49:40.387434 (+ 582us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:49:40.387597 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:40.387652 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 12:49:40.387747 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:49:40.398166 30032 raft_consensus.cc:1261] T 364c32e124544289a8eecea049ee08a0 P 192e00576a964e7aa82d94d918b9e5ef [term 1 FOLLOWER]: Refusing update from remote peer 25f13b7984a7401ab3623c90a9efa2b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:49:40.398166 30255 raft_consensus.cc:1261] T 364c32e124544289a8eecea049ee08a0 P bff68f2564244bab97d628c2ecc3c156 [term 1 FOLLOWER]: Refusing update from remote peer 25f13b7984a7401ab3623c90a9efa2b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:49:40.399255 30396 consensus_queue.cc:1035] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff68f2564244bab97d628c2ecc3c156" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 33647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:49:40.399603 30377 consensus_queue.cc:1035] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "192e00576a964e7aa82d94d918b9e5ef" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 41945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 12:49:40.404917 30336 tablet.cc:2369] T 364c32e124544289a8eecea049ee08a0 P bff68f2564244bab97d628c2ecc3c156: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:49:40.415354 30400 mvcc.cc:205] Tried to move back new op lower bound from 7317309769298771968 to 7317309769006493696. Current Snapshot: MvccSnapshot[applied={T|T < 7317309769298771968}]
I20260811 12:49:40.431618 30402 mvcc.cc:205] Tried to move back new op lower bound from 7317309769298771968 to 7317309769006493696. Current Snapshot: MvccSnapshot[applied={T|T < 7317309769298771968}]
I20260811 12:49:40.439985 29501 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:49:40.440299 29501 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:49:40.444774 29501 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c42b56beae984aeab6fda7f7b76d8d96: Sending DeleteTablet for 3 replicas of tablet 364c32e124544289a8eecea049ee08a0
I20260811 12:49:40.446127 29732 tablet_service.cc:1514] Processing DeleteTablet for tablet 364c32e124544289a8eecea049ee08a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:49:40 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:55062
I20260811 12:49:40.446159 30223 tablet_service.cc:1514] Processing DeleteTablet for tablet 364c32e124544289a8eecea049ee08a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:49:40 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:45960
I20260811 12:49:40.447551 29994 tablet_service.cc:1514] Processing DeleteTablet for tablet 364c32e124544289a8eecea049ee08a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:49:40 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:51764
I20260811 12:49:40.456499 30420 tablet_replica.cc:330] T 364c32e124544289a8eecea049ee08a0 P 192e00576a964e7aa82d94d918b9e5ef: stopping tablet replica
I20260811 12:49:40.457787 30420 raft_consensus.cc:2229] T 364c32e124544289a8eecea049ee08a0 P 192e00576a964e7aa82d94d918b9e5ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:49:40.458979 30420 raft_consensus.cc:2258] T 364c32e124544289a8eecea049ee08a0 P 192e00576a964e7aa82d94d918b9e5ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:49:40.459203 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 29566
I20260811 12:49:40.462515 30420 ts_tablet_manager.cc:1905] T 364c32e124544289a8eecea049ee08a0 P 192e00576a964e7aa82d94d918b9e5ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:49:40.463512 30418 tablet_replica.cc:330] T 364c32e124544289a8eecea049ee08a0 P bff68f2564244bab97d628c2ecc3c156: stopping tablet replica
I20260811 12:49:40.463979 30418 raft_consensus.cc:2229] T 364c32e124544289a8eecea049ee08a0 P bff68f2564244bab97d628c2ecc3c156 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:49:40.464501 30418 raft_consensus.cc:2258] T 364c32e124544289a8eecea049ee08a0 P bff68f2564244bab97d628c2ecc3c156 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:49:40.466672 30417 tablet_replica.cc:330] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0: stopping tablet replica
I20260811 12:49:40.467324 30417 raft_consensus.cc:2229] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:49:40.469102 30417 raft_consensus.cc:2258] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:49:40.471442 30417 ts_tablet_manager.cc:1905] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:49:40.474237 30418 ts_tablet_manager.cc:1905] T 364c32e124544289a8eecea049ee08a0 P bff68f2564244bab97d628c2ecc3c156: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:49:40.480754 30364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:40.461165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55074 (local address 127.7.111.65:40037)
0811 12:49:40.461438 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:40.461445 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:40.461469 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:49:40.462293 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:40.462299 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:40.462408 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:40.462536 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:40.464466 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.466145 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.468042 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.468695 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.468822 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:40.472507 (+ 3685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:40.472522 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:40.472540 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:40.472592 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:40.477094 (+ 4502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:40.478088 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:40.478097 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:40.478101 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:40.478374 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:40.478862 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:40.478867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:40.478870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:40.479356 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:40.479685 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:40.480071 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 12:49:40.480385 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260811 12:49:40.481534 30420 ts_tablet_manager.cc:1918] T 364c32e124544289a8eecea049ee08a0 P 192e00576a964e7aa82d94d918b9e5ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:49:40.481863 30420 log.cc:1196] T 364c32e124544289a8eecea049ee08a0 P 192e00576a964e7aa82d94d918b9e5ef: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/364c32e124544289a8eecea049ee08a0
I20260811 12:49:40.481933 29838 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:49:40.482640 30420 ts_tablet_manager.cc:1939] T 364c32e124544289a8eecea049ee08a0 P 192e00576a964e7aa82d94d918b9e5ef: Deleting consensus metadata
I20260811 12:49:40.483891 30418 ts_tablet_manager.cc:1918] T 364c32e124544289a8eecea049ee08a0 P bff68f2564244bab97d628c2ecc3c156: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:49:40.484047 30418 log.cc:1196] T 364c32e124544289a8eecea049ee08a0 P bff68f2564244bab97d628c2ecc3c156: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/364c32e124544289a8eecea049ee08a0
I20260811 12:49:40.484596 30418 ts_tablet_manager.cc:1939] T 364c32e124544289a8eecea049ee08a0 P bff68f2564244bab97d628c2ecc3c156: Deleting consensus metadata
I20260811 12:49:40.486454 29479 catalog_manager.cc:4841] TS bff68f2564244bab97d628c2ecc3c156 (127.7.111.67:33647): tablet 364c32e124544289a8eecea049ee08a0 (table test-table [id=9f9f44e3c72d402ea07e0419be61f5a5]) successfully deleted
I20260811 12:49:40.487792 29475 catalog_manager.cc:4841] TS 192e00576a964e7aa82d94d918b9e5ef (127.7.111.66:41945): tablet 364c32e124544289a8eecea049ee08a0 (table test-table [id=9f9f44e3c72d402ea07e0419be61f5a5]) successfully deleted
I20260811 12:49:40.589184 30417 ts_tablet_manager.cc:1918] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:49:40.589600 30417 log.cc:1196] T 364c32e124544289a8eecea049ee08a0 P 25f13b7984a7401ab3623c90a9efa2b0: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/364c32e124544289a8eecea049ee08a0
I20260811 12:49:40.589929 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 29566
I20260811 12:49:40.615558 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 29860
W20260811 12:49:40.615644 29475 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.111.65:40037 (error 108)
I20260811 12:49:40.617726 30424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:40.616907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:40037 (local address 127.0.0.1:55088)
0811 12:49:40.617324 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:40.617497 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":182,"threads_started":1}
W20260811 12:49:40.617970 29475 catalog_manager.cc:4584] TS 25f13b7984a7401ab3623c90a9efa2b0 (127.7.111.65:40037): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 364c32e124544289a8eecea049ee08a0: Network error: Client connection negotiation failed: client connection to 127.7.111.65:40037: connect: Connection refused (error 111)
I20260811 12:49:40.628654 30366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:40.616374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51774 (local address 127.7.111.66:41945)
0811 12:49:40.616614 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:40.616619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:40.616690 (+ 71us) server_negotiation.cc:407] Connection header received
0811 12:49:40.616947 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:40.616953 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:40.617046 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:40.617160 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:40.618721 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.619416 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.620652 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.621032 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.621119 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:40.623706 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:40.623719 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:40.623734 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:40.623770 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:40.626378 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:40.626984 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:40.626990 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:40.626993 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:40.627124 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:40.627405 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:40.627408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:40.627410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:40.627744 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:40.627962 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:40.628159 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 12:49:40.628339 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260811 12:49:40.629284 30104 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:49:40.671463 30424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:40.671004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:40037 (local address 127.0.0.1:55094)
0811 12:49:40.671214 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:40.671320 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 12:49:40.672470 30424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:40.672102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:40037 (local address 127.0.0.1:55096)
0811 12:49:40.672251 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:40.672337 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 12:49:40.717140 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 29860
I20260811 12:49:40.732941 30424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:40.732486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:40037 (local address 127.0.0.1:55110)
0811 12:49:40.732701 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:40.732789 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 12:49:40.734157 30424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:40.733685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:40037 (local address 127.0.0.1:55126)
0811 12:49:40.733841 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:40.733946 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 12:49:40.740814 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 30117
I20260811 12:49:40.753690 30363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:40.741533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45966 (local address 127.7.111.67:33647)
0811 12:49:40.741721 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:40.741726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:40.741755 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:49:40.741876 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:40.741880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:40.741954 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:40.742038 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:40.743323 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.744017 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.745340 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:40.745779 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:40.745889 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:40.748276 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:40.748292 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:40.748310 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:40.748341 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:40.750877 (+ 2536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:40.751505 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:40.751512 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:40.751515 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:40.751646 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:40.752004 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:40.752010 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:40.752012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:40.752397 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:40.752667 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:40.752975 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 12:49:40.753179 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:49:40.754292 30324 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:49:40.814874 30424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:40.814322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:40037 (local address 127.0.0.1:55138)
0811 12:49:40.814568 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:40.814681 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 12:49:40.815980 30424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:40.815515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:40037 (local address 127.0.0.1:55140)
0811 12:49:40.815727 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:40.815806 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260811 12:49:40.841514 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 30117
I20260811 12:49:40.862010 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 29447
I20260811 12:49:40.863207 29551 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:49:40.975611 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 29447
I20260811 12:49:41.013561 7613 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 12:49:41.358515 7613 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 12:49:41.358762 7613 mini_ranger.cc:66] Stopping Ranger...
I20260811 12:49:41.705030 7613 mini_ranger.cc:68] Stopped Ranger
2026-08-11 12:49:41.705 UTC [27809] LOG: received smart shutdown request
2026-08-11 12:49:41.708 UTC [27809] LOG: background worker "logical replication launcher" (PID 27823) exited with exit code 1
2026-08-11 12:49:41.708 UTC [27814] LOG: shutting down
2026-08-11 12:49:41.848 UTC [27809] LOG: database system is shut down
2026-08-11T12:49:41Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51849 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:49:42 dist-test-slave-f0tn krb5kdc[30444](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:49:42 dist-test-slave-f0tn krb5kdc[30444](info): set up 2 sockets
Aug 11 12:49:42 dist-test-slave-f0tn krb5kdc[30444](info): commencing operation
krb5kdc: starting...
W20260811 12:49:43.216727 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:49:43 dist-test-slave-f0tn krb5kdc[30444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:49:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:49:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:49:43.514205 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:34333
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:36699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:34333,127.7.111.125:35219,127.7.111.124:46497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:49:43.851017 30478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:43.851305 30478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:43.851437 30478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:49:43.858633 30478 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:49:43.858734 30478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:43.858960 30478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:43.859071 30478 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:49:43.859141 30478 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:49:43.868824 30478 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:36699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:34333,127.7.111.125:35219,127.7.111.124:46497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:34333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:43.870150 30478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:43.871500 30478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:43.882879 30489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:43.882944 30488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:43.882920 30495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:43.883651 30478 server_base.cc:1034] running on GCE node
I20260811 12:49:43.884357 30478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:43.885679 30478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:43.886876 30478 hybrid_clock.cc:648] HybridClock initialized: now 1786452583886836 us; error 31 us; skew 500 ppm
I20260811 12:49:43.887212 30478 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:43 dist-test-slave-f0tn krb5kdc[30444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452583, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:43.890249 30478 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:49:43.891707 30478 webserver.cc:460] Webserver started at http://127.7.111.126:45415/ using document root <none> and password file <none>
I20260811 12:49:43.892388 30478 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:43.892504 30478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:49:43.892791 30478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:49:43.895744 30478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "5d9a96db435d40c79570ed08bbd66e09"
format_stamp: "Formatted at 2026-08-11 12:49:43 on dist-test-slave-f0tn"
server_key: "7a25f4710801963a20fee524dfd77640"
server_key_iv: "1f686fbf46659a9b56935454224ff716"
server_key_version: "encryptionkey@0"
I20260811 12:49:43.896515 30478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "5d9a96db435d40c79570ed08bbd66e09"
format_stamp: "Formatted at 2026-08-11 12:49:43 on dist-test-slave-f0tn"
server_key: "7a25f4710801963a20fee524dfd77640"
server_key_iv: "1f686fbf46659a9b56935454224ff716"
server_key_version: "encryptionkey@0"
I20260811 12:49:43.901464 30478 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:49:43.904829 30503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:43.906088 30478 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 12:49:43.906235 30478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "5d9a96db435d40c79570ed08bbd66e09"
format_stamp: "Formatted at 2026-08-11 12:49:43 on dist-test-slave-f0tn"
server_key: "7a25f4710801963a20fee524dfd77640"
server_key_iv: "1f686fbf46659a9b56935454224ff716"
server_key_version: "encryptionkey@0"
I20260811 12:49:43.906409 30478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:43.923590 30478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:43.927827 30478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:43.928113 30478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:43.945636 30478 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:34333
I20260811 12:49:43.945668 30585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:34333 every 8 connection(s)
I20260811 12:49:43.946949 30478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:49:43.949898 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 30478
I20260811 12:49:43.950119 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:49:43.950904 30587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:49:43.950964 7613 external_mini_cluster.cc:1428] Setting key 500fde5b222bbc100ad4cf0ef5fd5c6a
I20260811 12:49:43.959285 30587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34333 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:49:43 dist-test-slave-f0tn krb5kdc[30444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452583, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.111.125@KRBTEST.COM; defaulting to no policy
I20260811 12:49:43.982273 30592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:43.962787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54352 (local address 127.7.111.126:34333)
0811 12:49:43.964423 (+ 1636us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:43.964431 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:43.964484 (+ 53us) server_negotiation.cc:407] Connection header received
0811 12:49:43.965977 (+ 1493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:43.965985 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:43.966273 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:43.966478 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:43.967611 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:43.969350 (+ 1739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:43.970731 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:43.971581 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:43.971745 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:43.975250 (+ 3505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:43.975261 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:43.975275 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:43.975313 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:43.978103 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:43.978949 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:43.978956 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:43.978958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:43.979099 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:43.979540 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:43.979543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:43.979545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:43.980013 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:43.980164 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:43.980839 (+ 675us) server_negotiation.cc:299] Negotiation successful
0811 12:49:43.981176 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":1417,"spinlock_wait_cycles":28928,"thread_start_us":1204,"threads_started":1}
I20260811 12:49:43.982292 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:43.962787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34333 (local address 127.0.0.1:54352)
0811 12:49:43.963617 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:43.963672 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:43.965606 (+ 1934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:43.966516 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:43.966521 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:43.967079 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:49:43.967408 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:43.967425 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:43.969528 (+ 2103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:43.969532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:43.970590 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:43.970602 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:43.971791 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:43.971796 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:43.971966 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:43.972554 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:49:43.972576 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:49:43.975021 (+ 2445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:49:43.978244 (+ 3223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:43.978258 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:43.978271 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:43.978829 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:43.979219 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:43.979224 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:43.979228 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:43.979442 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:43.980158 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:49:43.980167 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:49:43.980437 (+ 270us) client_negotiation.cc:769] Sending connection context
0811 12:49:43.980737 (+ 300us) client_negotiation.cc:240] Negotiation successful
0811 12:49:43.981322 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":163,"thread_start_us":362,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.125@KRBTEST.COM" created.
I20260811 12:49:43.985306 30587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:49:43.986245 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:43.985893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:35219 (local address 127.0.0.1:39392)
0811 12:49:43.986053 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:43.986142 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:35219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 12:49:43.986490 30507 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.111.125:35219: connect: Connection refused (error 111)
I20260811 12:49:43.987914 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:43.987583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:35219 (local address 127.0.0.1:39406)
0811 12:49:43.987737 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:43.987803 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:35219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 12:49:43.988128 30587 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.125:35219: Network error: Client connection negotiation failed: client connection to 127.7.111.125:35219: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.125.keytab.
Entry for principal kudu/127.7.111.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.125.keytab.
I20260811 12:49:44.014864 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.125:35219
--webserver_interface=127.7.111.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.125.keytab
--principal=kudu/127.7.111.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:36699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:34333,127.7.111.125:35219,127.7.111.124:46497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:49:44.037515 30587 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } attempt: 1
I20260811 12:49:44.038794 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.038342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:35219 (local address 127.0.0.1:39418)
0811 12:49:44.038539 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.038645 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:35219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 12:49:44.039804 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.039458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:35219 (local address 127.0.0.1:39430)
0811 12:49:44.039600 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.039668 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:35219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 12:49:44.039986 30587 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.125:35219: Network error: Client connection negotiation failed: client connection to 127.7.111.125:35219: connect: Connection refused (error 111)
I20260811 12:49:44.108201 30587 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } attempt: 2
I20260811 12:49:44.109465 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.108931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:35219 (local address 127.0.0.1:39434)
0811 12:49:44.109168 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.109276 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:35219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 12:49:44.110502 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.110106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:35219 (local address 127.0.0.1:39438)
0811 12:49:44.110253 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.110351 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:35219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 12:49:44.110673 30587 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.125:35219: Network error: Client connection negotiation failed: client connection to 127.7.111.125:35219: connect: Connection refused (error 111)
I20260811 12:49:44.201922 30587 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } attempt: 3
I20260811 12:49:44.202989 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.202546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:35219 (local address 127.0.0.1:39450)
0811 12:49:44.202746 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.202854 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:35219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 12:49:44.204039 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.203683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:35219 (local address 127.0.0.1:39452)
0811 12:49:44.203826 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.203895 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:35219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 12:49:44.204205 30587 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.125:35219: Network error: Client connection negotiation failed: client connection to 127.7.111.125:35219: connect: Connection refused (error 111)
I20260811 12:49:44.347460 30587 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } attempt: 4
I20260811 12:49:44.348656 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.348111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:35219 (local address 127.0.0.1:39460)
0811 12:49:44.348408 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.348497 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:35219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
W20260811 12:49:44.349291 30600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:44.349664 30600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:44.349825 30600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 12:49:44.349828 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.349369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:35219 (local address 127.0.0.1:39462)
0811 12:49:44.349549 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.349668 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:35219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 12:49:44.350025 30587 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.125:35219: Network error: Client connection negotiation failed: client connection to 127.7.111.125:35219: connect: Connection refused (error 111)
W20260811 12:49:44.357167 30600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:49:44.357263 30600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:44.357338 30600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:44.357438 30600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:49:44.357498 30600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:49:44.367355 30600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:36699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:34333,127.7.111.125:35219,127.7.111.124:46497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.125.keytab
--principal=kudu/127.7.111.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.125:35219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.111.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:44.368661 30600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:44.370054 30600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:44.381649 30606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:44.381585 30607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:44.382094 30610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:44.382642 30600 server_base.cc:1034] running on GCE node
I20260811 12:49:44.383360 30600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:44.384718 30600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:44.385913 30600 hybrid_clock.cc:648] HybridClock initialized: now 1786452584385870 us; error 29 us; skew 500 ppm
I20260811 12:49:44.386238 30600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:44 dist-test-slave-f0tn krb5kdc[30444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452584, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:44.389209 30600 init.cc:377] Logged in from keytab as kudu/127.7.111.125@KRBTEST.COM (short username kudu)
I20260811 12:49:44.390702 30600 webserver.cc:460] Webserver started at http://127.7.111.125:44821/ using document root <none> and password file <none>
I20260811 12:49:44.391362 30600 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:44.391472 30600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:49:44.391780 30600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:49:44.394730 30600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-1/data/instance:
uuid: "16bbb7d96de246af807188c576af1bb0"
format_stamp: "Formatted at 2026-08-11 12:49:44 on dist-test-slave-f0tn"
server_key: "6b188f3143961f071dc30571e9289ff9"
server_key_iv: "d874063e8c6c214c3d7b6a9aaf02a520"
server_key_version: "encryptionkey@0"
I20260811 12:49:44.395507 30600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-1/wal/instance:
uuid: "16bbb7d96de246af807188c576af1bb0"
format_stamp: "Formatted at 2026-08-11 12:49:44 on dist-test-slave-f0tn"
server_key: "6b188f3143961f071dc30571e9289ff9"
server_key_iv: "d874063e8c6c214c3d7b6a9aaf02a520"
server_key_version: "encryptionkey@0"
I20260811 12:49:44.400568 30600 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 12:49:44.403929 30623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:44.405246 30600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:49:44.405438 30600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-1/wal
uuid: "16bbb7d96de246af807188c576af1bb0"
format_stamp: "Formatted at 2026-08-11 12:49:44 on dist-test-slave-f0tn"
server_key: "6b188f3143961f071dc30571e9289ff9"
server_key_iv: "d874063e8c6c214c3d7b6a9aaf02a520"
server_key_version: "encryptionkey@0"
I20260811 12:49:44.405611 30600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:44.418102 30600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:44.421689 30600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:44.421947 30600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:44.438995 30600 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.125:35219
I20260811 12:49:44.439040 30715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.125:35219 every 8 connection(s)
I20260811 12:49:44.440284 30600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-1/data/info.pb
I20260811 12:49:44.440907 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 30600
I20260811 12:49:44.441090 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-1/wal/instance
I20260811 12:49:44.441632 7613 external_mini_cluster.cc:1428] Setting key 4132a51b69bc352d37e92f5bc302b5d3
I20260811 12:49:44.444038 30716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:49:44.450423 30716 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34333 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:49:44 dist-test-slave-f0tn krb5kdc[30444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452584, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.125@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.111.124@KRBTEST.COM; defaulting to no policy
I20260811 12:49:44.471117 30592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.452900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54358 (local address 127.7.111.126:34333)
0811 12:49:44.453104 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:44.453110 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:44.454417 (+ 1307us) server_negotiation.cc:407] Connection header received
0811 12:49:44.456313 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:44.456320 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:44.456422 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:44.456539 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:44.458019 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.459042 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.460543 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.461098 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.461212 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:44.464655 (+ 3443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:44.464661 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:44.464666 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:44.464718 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:44.468057 (+ 3339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:44.468763 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:44.468769 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:44.468773 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:44.468920 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:44.469331 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:44.469335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:44.469337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:44.469667 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:44.469820 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:44.470558 (+ 738us) server_negotiation.cc:299] Negotiation successful
0811 12:49:44.470804 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.124@KRBTEST.COM" created.
I20260811 12:49:44.472084 30720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.453338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34333 (local address 127.0.0.1:54358)
0811 12:49:44.454226 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.454280 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:44.456026 (+ 1746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:44.456723 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:44.456731 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:44.457248 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:49:44.457846 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:44.457864 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.459209 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.459214 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:44.460404 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:44.460417 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.461292 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.461297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:44.461527 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:44.462159 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:49:44.462190 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:49:44.464423 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:49:44.468195 (+ 3772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:44.468202 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:44.468215 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:44.468619 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:44.469010 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:44.469013 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:44.469015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:44.469200 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:44.469817 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:49:44.469824 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:49:44.470164 (+ 340us) client_negotiation.cc:769] Sending connection context
0811 12:49:44.470411 (+ 247us) client_negotiation.cc:240] Negotiation successful
0811 12:49:44.470898 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":383,"threads_started":1}
I20260811 12:49:44.473870 30716 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:49:44 dist-test-slave-f0tn krb5kdc[30444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452584, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.125@KRBTEST.COM for kudu/127.7.111.125@KRBTEST.COM
I20260811 12:49:44.488149 30720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.474504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:35219 (local address 127.0.0.1:39478)
0811 12:49:44.474719 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.474738 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:44.474843 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:44.475854 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:44.475859 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:44.476282 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:49:44.476426 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:44.476435 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.477821 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.477826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:44.478654 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:44.478666 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.479434 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.479439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:44.479639 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:44.480159 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:49:44.480184 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:49:44.482336 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:49:44.485909 (+ 3573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:44.485913 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:44.485916 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:44.486254 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:44.486629 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:44.486632 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:44.486634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:44.486782 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:44.487478 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:49:44.487481 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:49:44.487600 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 12:49:44.487715 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 12:49:44.487897 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 12:49:44.488483 30722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.474776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39478 (local address 127.7.111.125:35219)
0811 12:49:44.475458 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:44.475464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:44.475489 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:49:44.475529 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:44.475534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:44.475711 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:44.475846 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:44.476579 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.477695 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.478783 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.479301 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.479409 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:44.482540 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:44.482552 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:44.482565 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:44.482594 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:44.485796 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:44.486380 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:44.486389 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:44.486393 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:44.486531 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:44.486876 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:44.486881 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:44.486885 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:44.487365 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:44.487568 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:44.487854 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 12:49:44.488092 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":392,"threads_started":1}
I20260811 12:49:44.490836 30716 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:49:44.491998 30720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.491561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46497 (local address 127.0.0.1:44660)
0811 12:49:44.491735 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.491847 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 12:49:44.492276 30625 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
I20260811 12:49:44.493818 30720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.493502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46497 (local address 127.0.0.1:44666)
0811 12:49:44.493630 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.493715 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 12:49:44.494011 30716 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:46497: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.124.keytab.
Entry for principal kudu/127.7.111.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.124.keytab.
I20260811 12:49:44.506157 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.124:46497
--webserver_interface=127.7.111.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.124.keytab
--principal=kudu/127.7.111.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:36699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:34333,127.7.111.125:35219,127.7.111.124:46497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:49:44.543295 30716 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } attempt: 1
I20260811 12:49:44.544396 30720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.543940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46497 (local address 127.0.0.1:44678)
0811 12:49:44.544130 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.544243 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 12:49:44.545359 30720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.545014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46497 (local address 127.0.0.1:44684)
0811 12:49:44.545172 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.545245 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 12:49:44.545615 30716 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:46497: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
I20260811 12:49:44.613802 30716 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } attempt: 2
I20260811 12:49:44.614923 30720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.614472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46497 (local address 127.0.0.1:44700)
0811 12:49:44.614659 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.614766 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 12:49:44.615869 30720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.615525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46497 (local address 127.0.0.1:44714)
0811 12:49:44.615668 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.615749 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 12:49:44.616067 30716 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:46497: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
I20260811 12:49:44.649274 30587 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } attempt: 5
Aug 11 12:49:44 dist-test-slave-f0tn krb5kdc[30444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452583, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for kudu/127.7.111.125@KRBTEST.COM
I20260811 12:49:44.663483 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.650035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:35219 (local address 127.0.0.1:39482)
0811 12:49:44.650268 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.650289 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:44.650402 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:44.650832 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:44.650836 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:44.651209 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:49:44.651402 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:44.651412 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.652894 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.652900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:44.654033 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:44.654047 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.655031 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.655037 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:44.655188 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:44.655756 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:49:44.655775 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:49:44.657750 (+ 1975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:49:44.661248 (+ 3498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:44.661251 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:44.661254 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:44.661646 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:44.662031 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:44.662034 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:44.662036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:44.662166 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:44.662805 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:49:44.662808 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:49:44.662937 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 12:49:44.663036 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 12:49:44.663245 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 12:49:44.663786 30722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.650191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39482 (local address 127.7.111.125:35219)
0811 12:49:44.650411 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:44.650418 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:44.650473 (+ 55us) server_negotiation.cc:407] Connection header received
0811 12:49:44.650542 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:44.650548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:44.650647 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:44.650783 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:44.651578 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.652705 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.654198 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.654878 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.655049 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:44.657953 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:44.657959 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:44.657964 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:44.658006 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:44.661114 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:44.661761 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:44.661767 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:44.661771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:44.661921 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:44.662294 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:44.662299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:44.662303 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:44.662698 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:44.662856 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:44.663205 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 12:49:44.663442 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 12:49:44.664772 30587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:49:44.666016 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.665592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46497 (local address 127.0.0.1:46252)
0811 12:49:44.665787 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.665877 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 12:49:44.666920 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.666607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46497 (local address 127.0.0.1:46254)
0811 12:49:44.666748 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.666828 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 12:49:44.667114 30587 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:46497: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
I20260811 12:49:44.707258 30716 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } attempt: 3
I20260811 12:49:44.708365 30720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.707871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46497 (local address 127.0.0.1:46258)
0811 12:49:44.708092 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.708189 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 12:49:44.709568 30720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.709067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46497 (local address 127.0.0.1:46274)
0811 12:49:44.709301 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.709407 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
W20260811 12:49:44.709807 30716 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:46497: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
I20260811 12:49:44.718307 30587 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } attempt: 1
I20260811 12:49:44.719261 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.718871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46497 (local address 127.0.0.1:46288)
0811 12:49:44.719029 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.719120 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 12:49:44.720153 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.719878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46497 (local address 127.0.0.1:46294)
0811 12:49:44.719994 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.720057 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 12:49:44.720356 30587 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:46497: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
I20260811 12:49:44.788578 30587 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } attempt: 2
I20260811 12:49:44.789800 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.789287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46497 (local address 127.0.0.1:46308)
0811 12:49:44.789533 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.789621 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 12:49:44.790745 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.790454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46497 (local address 127.0.0.1:46318)
0811 12:49:44.790569 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.790632 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 12:49:44.790956 30587 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:46497: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
W20260811 12:49:44.846227 30729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:44.846527 30729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:44.846622 30729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 12:49:44.853022 30716 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } attempt: 4
W20260811 12:49:44.853904 30729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:49:44.853999 30729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:44.854084 30729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:44.854166 30729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
I20260811 12:49:44.854152 30720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.853642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46497 (local address 127.0.0.1:46326)
0811 12:49:44.853864 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.853954 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 12:49:44.854221 30729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:49:44.855365 30720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.854956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46497 (local address 127.0.0.1:46332)
0811 12:49:44.855117 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.855211 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 12:49:44.855571 30716 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:46497: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
I20260811 12:49:44.864595 30729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:36699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:34333,127.7.111.125:35219,127.7.111.124:46497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.124.keytab
--principal=kudu/127.7.111.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.124:46497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.111.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:44.865942 30729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:44.867197 30729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:44.878382 30747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:44.878439 30745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:44.878618 30743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:44.879741 30729 server_base.cc:1034] running on GCE node
I20260811 12:49:44.880475 30729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:44.881875 30729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:44.883080 30729 hybrid_clock.cc:648] HybridClock initialized: now 1786452584883028 us; error 43 us; skew 500 ppm
I20260811 12:49:44.883406 30729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:44 dist-test-slave-f0tn krb5kdc[30444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452584, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:44.886313 30729 init.cc:377] Logged in from keytab as kudu/127.7.111.124@KRBTEST.COM (short username kudu)
I20260811 12:49:44.887665 30729 webserver.cc:460] Webserver started at http://127.7.111.124:37099/ using document root <none> and password file <none>
I20260811 12:49:44.888353 30729 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:44.888466 30729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:49:44.888756 30729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:49:44.891820 30729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-2/data/instance:
uuid: "00faef5b8f884cdaa9c0b5566299392f"
format_stamp: "Formatted at 2026-08-11 12:49:44 on dist-test-slave-f0tn"
server_key: "adb07ca3590f251b77f3c9ceecd6f04c"
server_key_iv: "a43906751586b02908003428c77a0cea"
server_key_version: "encryptionkey@0"
I20260811 12:49:44.892685 30729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-2/wal/instance:
uuid: "00faef5b8f884cdaa9c0b5566299392f"
format_stamp: "Formatted at 2026-08-11 12:49:44 on dist-test-slave-f0tn"
server_key: "adb07ca3590f251b77f3c9ceecd6f04c"
server_key_iv: "a43906751586b02908003428c77a0cea"
server_key_version: "encryptionkey@0"
I20260811 12:49:44.897157 30587 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } attempt: 3
I20260811 12:49:44.897964 30729 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:49:44.898190 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.897727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46497 (local address 127.0.0.1:46334)
0811 12:49:44.897946 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.898025 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 12:49:44.899096 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.898754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46497 (local address 127.0.0.1:46340)
0811 12:49:44.898928 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.899001 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260811 12:49:44.899307 30587 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:46497: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46497: connect: Connection refused (error 111)
I20260811 12:49:44.901412 30760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:44.902794 30729 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:49:44.902957 30729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-2/wal
uuid: "00faef5b8f884cdaa9c0b5566299392f"
format_stamp: "Formatted at 2026-08-11 12:49:44 on dist-test-slave-f0tn"
server_key: "adb07ca3590f251b77f3c9ceecd6f04c"
server_key_iv: "a43906751586b02908003428c77a0cea"
server_key_version: "encryptionkey@0"
I20260811 12:49:44.903131 30729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:44.928253 30729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:44.932070 30729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:44.932345 30729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:44.958662 30729 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.124:46497
I20260811 12:49:44.958693 30838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.124:46497 every 8 connection(s)
I20260811 12:49:44.959918 30729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-2/data/info.pb
I20260811 12:49:44.963501 30839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:49:44.967037 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 30729
I20260811 12:49:44.967218 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786452375851340-7613-0/minicluster-data/master-2/wal/instance
I20260811 12:49:44.967724 7613 external_mini_cluster.cc:1428] Setting key 879a568973250f315dd9e3e4c6fcda66
I20260811 12:49:44.970304 30839 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34333 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:49:44 dist-test-slave-f0tn krb5kdc[30444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.124@KRBTEST.COM
Aug 11 12:49:44 dist-test-slave-f0tn krb5kdc[30444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452584, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.124@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:49:44.989037 30843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.974162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39024 (local address 127.7.111.126:34333)
0811 12:49:44.974694 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:44.974700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:44.975135 (+ 435us) server_negotiation.cc:407] Connection header received
0811 12:49:44.975310 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:44.975314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:44.975374 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:44.975460 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:44.976656 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.977568 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.979149 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.979554 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.979651 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:44.983664 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:44.983669 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:44.983672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:44.983704 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:44.986321 (+ 2617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:44.986936 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:44.986941 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:44.986943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:44.987064 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:44.987461 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:44.987464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:44.987466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:44.987717 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:44.987825 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:44.988442 (+ 617us) server_negotiation.cc:299] Negotiation successful
0811 12:49:44.988661 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":273,"threads_started":1}
I20260811 12:49:44.990202 30844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.974147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34333 (local address 127.0.0.1:39024)
0811 12:49:44.975001 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.975043 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:44.975196 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:44.975554 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:44.975561 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:44.976179 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:49:44.976501 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:44.976515 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.977728 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.977734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:44.978998 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:44.979013 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.979789 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.979799 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:44.979970 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:44.980573 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:49:44.980605 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:49:44.983480 (+ 2875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:49:44.986444 (+ 2964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:44.986453 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:44.986462 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:44.986819 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:44.987169 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:44.987187 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:44.987191 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:44.987352 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:44.987838 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:49:44.987845 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:49:44.988041 (+ 196us) client_negotiation.cc:769] Sending connection context
0811 12:49:44.988331 (+ 290us) client_negotiation.cc:240] Negotiation successful
0811 12:49:44.988785 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"mutex_wait_us":36,"thread_start_us":503,"threads_started":1}
I20260811 12:49:44.990222 30842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.969543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46346 (local address 127.7.111.124:46497)
0811 12:49:44.970281 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:44.970296 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:44.970338 (+ 42us) server_negotiation.cc:407] Connection header received
0811 12:49:44.972297 (+ 1959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:44.972320 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:44.972747 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:44.973090 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:44.974861 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.976343 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.977888 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.978603 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.978732 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:44.983388 (+ 4656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:44.983400 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:44.983415 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:44.983454 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:44.986187 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:44.986918 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:44.986927 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:44.986930 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:44.987074 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:44.987416 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:44.987422 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:44.987425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:44.987981 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:44.988267 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:44.988544 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 12:49:44.989088 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":148,"server-negotiator.queue_time_us":544,"thread_start_us":328,"threads_started":1}
I20260811 12:49:44.991920 30839 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:49:44 dist-test-slave-f0tn krb5kdc[30444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452584, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.124@KRBTEST.COM for kudu/127.7.111.125@KRBTEST.COM
Aug 11 12:49:44 dist-test-slave-f0tn krb5kdc[30444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:49:45.005317 30844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.992572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:35219 (local address 127.0.0.1:33896)
0811 12:49:44.992728 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:44.992745 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:44.992830 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:44.993474 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:44.993478 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:44.993903 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:49:44.994118 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:44.994130 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.995203 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.995209 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:44.996195 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:44.996210 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.996889 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.996895 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:44.997125 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:44.997706 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:49:44.997727 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:49:44.999900 (+ 2173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:49:45.003044 (+ 3144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:45.003049 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:45.003054 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:45.003406 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:45.003774 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:45.003780 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:45.003783 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:45.003919 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:45.004404 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:49:45.004408 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:49:45.004548 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 12:49:45.004666 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 12:49:45.004958 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 12:49:45.005340 30722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.992819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33896 (local address 127.7.111.125:35219)
0811 12:49:44.993006 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:44.993011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:44.993025 (+ 14us) server_negotiation.cc:407] Connection header received
0811 12:49:44.993080 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:44.993085 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:44.993162 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:44.993267 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:44.994272 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.994984 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.996341 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.996757 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.996844 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:45.000081 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:45.000086 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:45.000090 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:45.000123 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:45.002917 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:45.003515 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:45.003519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:45.003522 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:45.003669 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:45.004020 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:45.004024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:45.004026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:45.004292 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:45.004419 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:45.004772 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 12:49:45.004980 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:49:45.006259 30839 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:49:45.007658 30843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:44.993250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39034 (local address 127.7.111.126:34333)
0811 12:49:44.993486 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:44.993490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:44.993504 (+ 14us) server_negotiation.cc:407] Connection header received
0811 12:49:44.993619 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:44.993622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:44.993686 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:44.993758 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:44.995076 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.995759 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.997167 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:44.997619 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:44.997715 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:45.002143 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:45.002147 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:45.002150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:45.002190 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:45.004723 (+ 2533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:45.005349 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:45.005355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:45.005358 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:45.005714 (+ 356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:45.006217 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:45.006221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:45.006223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:45.006541 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:45.006802 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:45.007144 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 12:49:45.007382 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
Aug 11 12:49:45 dist-test-slave-f0tn krb5kdc[30444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452584, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.124@KRBTEST.COM for kudu/127.7.111.124@KRBTEST.COM
Aug 11 12:49:45 dist-test-slave-f0tn krb5kdc[30444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.125@KRBTEST.COM
I20260811 12:49:45.019145 30844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:45.007009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46497 (local address 127.0.0.1:46356)
0811 12:49:45.007278 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:45.007296 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:45.007399 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:45.007814 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:45.007818 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:45.008413 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:49:45.008577 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:45.008586 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:45.009690 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:45.009695 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:45.010584 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:45.010596 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:45.011242 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:45.011247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:45.011428 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:45.011907 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:49:45.011924 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:49:45.013935 (+ 2011us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:49:45.016723 (+ 2788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:45.016729 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:45.016733 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:45.017180 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:45.017595 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:45.017600 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:45.017604 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:45.017770 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:45.018349 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:49:45.018352 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:49:45.018469 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 12:49:45.018584 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 12:49:45.018881 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166}
I20260811 12:49:45.019414 30842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:45.006978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46356 (local address 127.7.111.124:46497)
0811 12:49:45.007139 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:45.007144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:45.007361 (+ 217us) server_negotiation.cc:407] Connection header received
0811 12:49:45.007502 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:45.007508 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:45.007577 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:45.007666 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:45.008709 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:45.009579 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:45.010712 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:45.011109 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:45.011212 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:45.014110 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:45.014114 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:45.014117 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:45.014144 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:45.016600 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:45.017300 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:45.017304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:45.017306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:45.017487 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:45.017872 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:45.017875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:45.017877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:45.018234 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:45.018379 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:45.018709 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 12:49:45.019052 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":107,"server-negotiator.queue_time_us":43}
I20260811 12:49:45.023558 30722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:45.009209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33900 (local address 127.7.111.125:35219)
0811 12:49:45.009339 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:45.009344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:45.009608 (+ 264us) server_negotiation.cc:407] Connection header received
0811 12:49:45.009783 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:45.009789 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:45.009850 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:45.009924 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:45.011297 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:45.012000 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:45.013401 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:45.013838 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:45.013933 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:45.018027 (+ 4094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:45.018033 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:45.018037 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:45.018074 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:45.021120 (+ 3046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:45.021918 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:45.021922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:45.021924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:45.022057 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:45.022401 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:45.022404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:45.022406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:45.022636 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:45.022833 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:45.023137 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 12:49:45.023312 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 12:49:45.025125 30839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f: Bootstrap starting.
I20260811 12:49:45.028254 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:49:45.028777 30839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f: Neither blocks nor log segments found. Creating new log.
I20260811 12:49:45.030028 30839 log.cc:824] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f: Log is configured to *not* fsync() on all Append() calls
I20260811 12:49:45.033229 30839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f: No bootstrap required, opened a new log
I20260811 12:49:45.044601 30839 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d9a96db435d40c79570ed08bbd66e09" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34333 } } peers { permanent_uuid: "16bbb7d96de246af807188c576af1bb0" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } } peers { permanent_uuid: "00faef5b8f884cdaa9c0b5566299392f" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } }
I20260811 12:49:45.045003 30839 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:49:45.045121 30839 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00faef5b8f884cdaa9c0b5566299392f, State: Initialized, Role: FOLLOWER
I20260811 12:49:45.045909 30839 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d9a96db435d40c79570ed08bbd66e09" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34333 } } peers { permanent_uuid: "16bbb7d96de246af807188c576af1bb0" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } } peers { permanent_uuid: "00faef5b8f884cdaa9c0b5566299392f" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } }
I20260811 12:49:45.052497 30843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:45.033838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39050 (local address 127.7.111.126:34333)
0811 12:49:45.034008 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:45.034012 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:45.035055 (+ 1043us) server_negotiation.cc:407] Connection header received
0811 12:49:45.035368 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:45.035373 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:45.035450 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:45.035516 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:45.037702 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:45.038761 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:45.040925 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:45.041610 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:45.041702 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:45.045578 (+ 3876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:45.045585 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:45.045589 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:45.045627 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:45.049057 (+ 3430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:45.050002 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:45.050008 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:45.050012 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:45.050172 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:45.050660 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:45.050665 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:45.050669 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:45.051033 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:45.051298 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:45.051850 (+ 552us) server_negotiation.cc:299] Negotiation successful
0811 12:49:45.052084 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 12:49:45.054868 30839 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:49:45.055101 30842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:45.034347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46366 (local address 127.7.111.124:46497)
0811 12:49:45.034542 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:45.034547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:45.035338 (+ 791us) server_negotiation.cc:407] Connection header received
0811 12:49:45.035605 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:45.035610 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:45.035702 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:45.035773 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:45.038169 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:45.039374 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:45.041669 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:45.042484 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:45.042584 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:45.047090 (+ 4506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:45.047096 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:45.047101 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:45.047139 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:45.050530 (+ 3391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:45.051393 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:45.051399 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:45.051402 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:45.051555 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:45.052045 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:45.052050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:45.052054 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:45.052484 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:45.052959 (+ 475us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:45.053573 (+ 614us) server_negotiation.cc:299] Negotiation successful
0811 12:49:45.054653 (+ 1080us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:49:45.053263 30857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d9a96db435d40c79570ed08bbd66e09" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34333 } } peers { permanent_uuid: "16bbb7d96de246af807188c576af1bb0" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } } peers { permanent_uuid: "00faef5b8f884cdaa9c0b5566299392f" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } } }
I20260811 12:49:45.056350 30857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:49:45.061250 30722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:45.035723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33916 (local address 127.7.111.125:35219)
0811 12:49:45.035882 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:45.035887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:45.045939 (+ 10052us) server_negotiation.cc:407] Connection header received
0811 12:49:45.046134 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:45.046139 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:45.046218 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:45.046303 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:45.047831 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:45.048894 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:45.050908 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:45.051512 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:45.051602 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:45.055080 (+ 3478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:45.055086 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:45.055090 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:45.055133 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:45.058586 (+ 3453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:45.059394 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:45.059398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:45.059400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:45.059525 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:45.059880 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:45.059884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:45.059886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:45.060166 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:45.060403 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:45.060702 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 12:49:45.060919 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 12:49:45.065208 30875 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:49:45.066170 30875 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:49:45.063007 30729 master_runner.cc:186] Error getting master registration for 127.7.111.126:34333: OK, instance_id { permanent_uuid: "5d9a96db435d40c79570ed08bbd66e09" instance_seqno: 1786452583940727 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:49:45.071240 30729 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 12:49:45.076491 30587 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } attempt: 4
Aug 11 12:49:45 dist-test-slave-f0tn krb5kdc[30444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452583, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for kudu/127.7.111.124@KRBTEST.COM
I20260811 12:49:45.090530 30842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:45.077208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46382 (local address 127.7.111.124:46497)
0811 12:49:45.077406 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:45.077412 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:45.077431 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:49:45.077484 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:45.077488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:45.077596 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:45.077695 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:45.078764 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:45.079500 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:45.080720 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:45.081220 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:45.081328 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:45.084221 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:45.084225 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:45.084228 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:45.084258 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:45.086982 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:45.088626 (+ 1644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:45.088632 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:45.088636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:45.088800 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:45.089232 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:45.089236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:45.089238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:45.089559 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:45.089667 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:45.090090 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 12:49:45.090274 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:49:45.090696 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:45.077016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46497 (local address 127.0.0.1:46382)
0811 12:49:45.077210 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:45.077231 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:45.077335 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:45.077892 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:45.077897 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:45.078419 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:49:45.078621 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:45.078632 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:45.079626 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:45.079631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:45.080579 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:45.080591 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:45.081364 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:45.081370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:45.081584 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:45.082129 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:49:45.082147 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:49:45.084037 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:49:45.087126 (+ 3089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:45.087132 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:45.087136 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:45.087696 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:45.088927 (+ 1231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:45.088932 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:45.088936 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:45.089114 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:45.089692 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:49:45.089697 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:49:45.089867 (+ 170us) client_negotiation.cc:769] Sending connection context
0811 12:49:45.090031 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 12:49:45.090307 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 12:49:45.095938 30587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09: Bootstrap starting.
I20260811 12:49:45.099185 30587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09: Neither blocks nor log segments found. Creating new log.
I20260811 12:49:45.100220 30587 log.cc:824] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09: Log is configured to *not* fsync() on all Append() calls
I20260811 12:49:45.103006 30587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09: No bootstrap required, opened a new log
I20260811 12:49:45.111439 30587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d9a96db435d40c79570ed08bbd66e09" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34333 } } peers { permanent_uuid: "16bbb7d96de246af807188c576af1bb0" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } } peers { permanent_uuid: "00faef5b8f884cdaa9c0b5566299392f" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } }
I20260811 12:49:45.111737 30587 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:49:45.111855 30587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d9a96db435d40c79570ed08bbd66e09, State: Initialized, Role: FOLLOWER
I20260811 12:49:45.112450 30587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d9a96db435d40c79570ed08bbd66e09" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34333 } } peers { permanent_uuid: "16bbb7d96de246af807188c576af1bb0" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } } peers { permanent_uuid: "00faef5b8f884cdaa9c0b5566299392f" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } }
I20260811 12:49:45.113328 30879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d9a96db435d40c79570ed08bbd66e09" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34333 } } peers { permanent_uuid: "16bbb7d96de246af807188c576af1bb0" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } } peers { permanent_uuid: "00faef5b8f884cdaa9c0b5566299392f" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } } }
I20260811 12:49:45.113737 30879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:49:45.114531 30587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 12:49:45.124581 30898 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:49:45.124709 30898 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:49:45.124038 30478 master_runner.cc:186] Error getting master registration for 127.7.111.125:35219: OK, instance_id { permanent_uuid: "16bbb7d96de246af807188c576af1bb0" instance_seqno: 1786452584434079 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:49:45.135044 30478 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 12:49:45.138867 30879 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:49:45.139151 30879 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d9a96db435d40c79570ed08bbd66e09" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34333 } } peers { permanent_uuid: "16bbb7d96de246af807188c576af1bb0" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } } peers { permanent_uuid: "00faef5b8f884cdaa9c0b5566299392f" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } }
I20260811 12:49:45.140864 30879 leader_election.cc:288] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16bbb7d96de246af807188c576af1bb0 (127.7.111.125:35219), 00faef5b8f884cdaa9c0b5566299392f (127.7.111.124:46497)
I20260811 12:49:45.141882 30799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5d9a96db435d40c79570ed08bbd66e09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00faef5b8f884cdaa9c0b5566299392f" is_pre_election: true
I20260811 12:49:45.142549 30799 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d9a96db435d40c79570ed08bbd66e09 in term 0.
I20260811 12:49:45.143219 30504 leader_election.cc:302] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00faef5b8f884cdaa9c0b5566299392f, 5d9a96db435d40c79570ed08bbd66e09; no voters:
I20260811 12:49:45.143682 30879 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:49:45.143836 30879 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:49:45.143931 30879 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:49:45.147358 30879 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d9a96db435d40c79570ed08bbd66e09" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34333 } } peers { permanent_uuid: "16bbb7d96de246af807188c576af1bb0" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } } peers { permanent_uuid: "00faef5b8f884cdaa9c0b5566299392f" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } }
I20260811 12:49:45.148937 30799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5d9a96db435d40c79570ed08bbd66e09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00faef5b8f884cdaa9c0b5566299392f"
I20260811 12:49:45.149163 30799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:49:45.152566 30799 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d9a96db435d40c79570ed08bbd66e09 in term 1.
I20260811 12:49:45.152839 30879 leader_election.cc:288] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [CANDIDATE]: Term 1 election: Requested vote from peers 16bbb7d96de246af807188c576af1bb0 (127.7.111.125:35219), 00faef5b8f884cdaa9c0b5566299392f (127.7.111.124:46497)
I20260811 12:49:45.153141 30504 leader_election.cc:302] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00faef5b8f884cdaa9c0b5566299392f, 5d9a96db435d40c79570ed08bbd66e09; no voters:
I20260811 12:49:45.153524 30879 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:49:45.154762 30716 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } attempt: 5
I20260811 12:49:45.165688 30879 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [term 1 LEADER]: Becoming Leader. State: Replica: 5d9a96db435d40c79570ed08bbd66e09, State: Running, Role: LEADER
I20260811 12:49:45.166652 30879 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d9a96db435d40c79570ed08bbd66e09" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34333 } } peers { permanent_uuid: "16bbb7d96de246af807188c576af1bb0" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } } peers { permanent_uuid: "00faef5b8f884cdaa9c0b5566299392f" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } }
Aug 11 12:49:45 dist-test-slave-f0tn krb5kdc[30444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452584, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.125@KRBTEST.COM for kudu/127.7.111.124@KRBTEST.COM
W20260811 12:49:45.180861 30513 tablet.cc:2369] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:49:45.184075 30720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:45.155337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46497 (local address 127.0.0.1:46384)
0811 12:49:45.155559 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:45.155578 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:45.155701 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:45.156431 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:45.156437 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:45.156980 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:49:45.157189 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:45.157203 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:45.158715 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:45.158720 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:45.160105 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:45.160123 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:45.161394 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:45.161401 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:45.161619 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:45.162321 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:49:45.162346 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:49:45.171450 (+ 9104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:49:45.178167 (+ 6717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:45.178200 (+ 33us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:45.178205 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:45.179133 (+ 928us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:45.179928 (+ 795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:45.179934 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:45.179938 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:45.180217 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:45.181596 (+ 1379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:49:45.181602 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:49:45.181935 (+ 333us) client_negotiation.cc:769] Sending connection context
0811 12:49:45.182185 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 12:49:45.182716 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125}
W20260811 12:49:45.184360 30768 tablet.cc:2369] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:49:45.186048 30902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d9a96db435d40c79570ed08bbd66e09. Latest consensus state: current_term: 1 leader_uuid: "5d9a96db435d40c79570ed08bbd66e09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d9a96db435d40c79570ed08bbd66e09" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34333 } } peers { permanent_uuid: "16bbb7d96de246af807188c576af1bb0" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } } peers { permanent_uuid: "00faef5b8f884cdaa9c0b5566299392f" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } } }
I20260811 12:49:45.186476 30902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [sys.catalog]: This master's current role is: LEADER
I20260811 12:49:45.187618 30904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:49:45.189579 30842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:45.155392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46384 (local address 127.7.111.124:46497)
0811 12:49:45.155559 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:45.155564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:45.155666 (+ 102us) server_negotiation.cc:407] Connection header received
0811 12:49:45.155832 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:45.155838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:45.155949 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:45.156223 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:45.157363 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:45.158498 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:45.160318 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:45.161019 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:45.161142 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:45.171761 (+ 10619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:45.171768 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:45.171773 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:45.171819 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:45.177924 (+ 6105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:45.179415 (+ 1491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:45.179421 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:45.179425 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:45.179737 (+ 312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:45.180481 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:45.180487 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:45.180490 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:45.181339 (+ 849us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:45.183212 (+ 1873us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:45.183851 (+ 639us) server_negotiation.cc:299] Negotiation successful
0811 12:49:45.188763 (+ 4912us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 12:49:45.190346 30904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:49:45.195601 30716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16bbb7d96de246af807188c576af1bb0: Bootstrap starting.
I20260811 12:49:45.206175 30799 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f [term 1 FOLLOWER]: Refusing update from remote peer 5d9a96db435d40c79570ed08bbd66e09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:49:45.207121 30716 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16bbb7d96de246af807188c576af1bb0: Neither blocks nor log segments found. Creating new log.
I20260811 12:49:45.207161 30902 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00faef5b8f884cdaa9c0b5566299392f" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:49:45.211021 30716 log.cc:824] T 00000000000000000000000000000000 P 16bbb7d96de246af807188c576af1bb0: Log is configured to *not* fsync() on all Append() calls
I20260811 12:49:45.217821 30716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16bbb7d96de246af807188c576af1bb0: No bootstrap required, opened a new log
I20260811 12:49:45.233290 30716 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16bbb7d96de246af807188c576af1bb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d9a96db435d40c79570ed08bbd66e09" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34333 } } peers { permanent_uuid: "16bbb7d96de246af807188c576af1bb0" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } } peers { permanent_uuid: "00faef5b8f884cdaa9c0b5566299392f" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } }
I20260811 12:49:45.233604 30857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d9a96db435d40c79570ed08bbd66e09. Latest consensus state: current_term: 1 leader_uuid: "5d9a96db435d40c79570ed08bbd66e09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d9a96db435d40c79570ed08bbd66e09" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34333 } } peers { permanent_uuid: "16bbb7d96de246af807188c576af1bb0" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } } peers { permanent_uuid: "00faef5b8f884cdaa9c0b5566299392f" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } } }
I20260811 12:49:45.233880 30716 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16bbb7d96de246af807188c576af1bb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:49:45.233971 30857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:49:45.234001 30716 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16bbb7d96de246af807188c576af1bb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16bbb7d96de246af807188c576af1bb0, State: Initialized, Role: FOLLOWER
I20260811 12:49:45.234670 30716 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16bbb7d96de246af807188c576af1bb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d9a96db435d40c79570ed08bbd66e09" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34333 } } peers { permanent_uuid: "16bbb7d96de246af807188c576af1bb0" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } } peers { permanent_uuid: "00faef5b8f884cdaa9c0b5566299392f" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } }
I20260811 12:49:45.239837 30909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16bbb7d96de246af807188c576af1bb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d9a96db435d40c79570ed08bbd66e09" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34333 } } peers { permanent_uuid: "16bbb7d96de246af807188c576af1bb0" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } } peers { permanent_uuid: "00faef5b8f884cdaa9c0b5566299392f" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } } }
I20260811 12:49:45.240808 30909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16bbb7d96de246af807188c576af1bb0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:49:45.241329 30716 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16bbb7d96de246af807188c576af1bb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:49:45.242233 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5d9a96db435d40c79570ed08bbd66e09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16bbb7d96de246af807188c576af1bb0"
I20260811 12:49:45.243731 30669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16bbb7d96de246af807188c576af1bb0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:49:45.247273 30907 mvcc.cc:205] Tried to move back new op lower bound from 7317309788992077824 to 7317309788848668672. Current Snapshot: MvccSnapshot[applied={T|T < 7317309788992077824}]
I20260811 12:49:45.247470 30669 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 16bbb7d96de246af807188c576af1bb0 [term 1 FOLLOWER]: Refusing update from remote peer 5d9a96db435d40c79570ed08bbd66e09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:49:45.248041 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5d9a96db435d40c79570ed08bbd66e09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16bbb7d96de246af807188c576af1bb0" is_pre_election: true
I20260811 12:49:45.248449 30671 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 16bbb7d96de246af807188c576af1bb0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d9a96db435d40c79570ed08bbd66e09 in term 1.
I20260811 12:49:45.249553 30879 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16bbb7d96de246af807188c576af1bb0" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:49:45.259840 30879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5d9a96db435d40c79570ed08bbd66e09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d9a96db435d40c79570ed08bbd66e09" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34333 } } peers { permanent_uuid: "16bbb7d96de246af807188c576af1bb0" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } } peers { permanent_uuid: "00faef5b8f884cdaa9c0b5566299392f" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } } }
I20260811 12:49:45.260169 30879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [sys.catalog]: This master's current role is: LEADER
I20260811 12:49:45.262833 30904 catalog_manager.cc:1347] Generated new cluster ID: eab3a55173a84719b64695a879a0addd
I20260811 12:49:45.262935 30904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:49:45.268225 30857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5d9a96db435d40c79570ed08bbd66e09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d9a96db435d40c79570ed08bbd66e09" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34333 } } peers { permanent_uuid: "16bbb7d96de246af807188c576af1bb0" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } } peers { permanent_uuid: "00faef5b8f884cdaa9c0b5566299392f" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } } }
I20260811 12:49:45.269340 30857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:49:45.285584 30904 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:49:45.287652 30904 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:49:45.298743 30909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16bbb7d96de246af807188c576af1bb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d9a96db435d40c79570ed08bbd66e09. Latest consensus state: current_term: 1 leader_uuid: "5d9a96db435d40c79570ed08bbd66e09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d9a96db435d40c79570ed08bbd66e09" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34333 } } peers { permanent_uuid: "16bbb7d96de246af807188c576af1bb0" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } } peers { permanent_uuid: "00faef5b8f884cdaa9c0b5566299392f" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } } }
I20260811 12:49:45.299121 30909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16bbb7d96de246af807188c576af1bb0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:49:45.299525 30928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16bbb7d96de246af807188c576af1bb0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5d9a96db435d40c79570ed08bbd66e09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d9a96db435d40c79570ed08bbd66e09" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34333 } } peers { permanent_uuid: "16bbb7d96de246af807188c576af1bb0" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } } peers { permanent_uuid: "00faef5b8f884cdaa9c0b5566299392f" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } } }
I20260811 12:49:45.299942 30928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16bbb7d96de246af807188c576af1bb0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:49:45.301087 30879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5d9a96db435d40c79570ed08bbd66e09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d9a96db435d40c79570ed08bbd66e09" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34333 } } peers { permanent_uuid: "16bbb7d96de246af807188c576af1bb0" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 35219 } } peers { permanent_uuid: "00faef5b8f884cdaa9c0b5566299392f" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46497 } } }
I20260811 12:49:45.301446 30879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09 [sys.catalog]: This master's current role is: LEADER
I20260811 12:49:45.331604 30904 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d9a96db435d40c79570ed08bbd66e09: Generated new TSK 0
I20260811 12:49:45.332568 30904 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:49:45.354410 30942 catalog_manager.cc:1259] Loaded cluster ID: eab3a55173a84719b64695a879a0addd
I20260811 12:49:45.354530 30942 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 16bbb7d96de246af807188c576af1bb0: loading cluster ID for follower catalog manager: success
I20260811 12:49:45.358544 30942 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 16bbb7d96de246af807188c576af1bb0: acquiring CA information for follower catalog manager: success
I20260811 12:49:45.361074 30942 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 16bbb7d96de246af807188c576af1bb0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:49:46.068212 30875 catalog_manager.cc:1259] Loaded cluster ID: eab3a55173a84719b64695a879a0addd
I20260811 12:49:46.068320 30875 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f: loading cluster ID for follower catalog manager: success
I20260811 12:49:46.070811 30875 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f: acquiring CA information for follower catalog manager: success
I20260811 12:49:46.072618 30875 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 00faef5b8f884cdaa9c0b5566299392f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:49:46.506326 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 30478
I20260811 12:49:46.507727 30579 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:49:46.620961 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 30478
I20260811 12:49:46.650764 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 30600
I20260811 12:49:46.651849 30704 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:49:46.759014 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 30600
I20260811 12:49:46.787945 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 30729
I20260811 12:49:46.789016 30831 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:49:46.895886 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 30729
2026-08-11T12:49:46Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4939 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:49:47 dist-test-slave-f0tn krb5kdc[30961](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:49:47 dist-test-slave-f0tn krb5kdc[30961](info): set up 2 sockets
Aug 11 12:49:47 dist-test-slave-f0tn krb5kdc[30961](info): commencing operation
krb5kdc: starting...
W20260811 12:49:48.146504 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:49:48 dist-test-slave-f0tn krb5kdc[30961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:49:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:49:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:49:48.448452 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:38457
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:38629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:38457
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:49:48.791606 30982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:48.791882 30982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:48.791972 30982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:49:48.799151 30982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:49:48.799259 30982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:48.799304 30982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:48.799358 30982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:49:48.799401 30982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:49:48.809278 30982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:38629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:38457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:38457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:48.810621 30982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:48.811965 30982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:48.824056 30987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:48.823990 30989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:48.824011 30992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:48.824913 30982 server_base.cc:1034] running on GCE node
I20260811 12:49:48.825734 30982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:48.827139 30982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:48.828365 30982 hybrid_clock.cc:648] HybridClock initialized: now 1786452588828310 us; error 36 us; skew 500 ppm
I20260811 12:49:48.828706 30982 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:48 dist-test-slave-f0tn krb5kdc[30961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452588, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:48.836339 30982 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:49:48.837895 30982 webserver.cc:460] Webserver started at http://127.7.111.126:44101/ using document root <none> and password file <none>
I20260811 12:49:48.838603 30982 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:48.838729 30982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:49:48.839084 30982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:49:48.842278 30982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "753adaf7a4a44a71bda2e45051ca1c47"
format_stamp: "Formatted at 2026-08-11 12:49:48 on dist-test-slave-f0tn"
server_key: "c1d9d8724f0b3c6e4fcfa68bc50d10c5"
server_key_iv: "ff723e185c9e5f32353c1193839ba6db"
server_key_version: "encryptionkey@0"
I20260811 12:49:48.843088 30982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "753adaf7a4a44a71bda2e45051ca1c47"
format_stamp: "Formatted at 2026-08-11 12:49:48 on dist-test-slave-f0tn"
server_key: "c1d9d8724f0b3c6e4fcfa68bc50d10c5"
server_key_iv: "ff723e185c9e5f32353c1193839ba6db"
server_key_version: "encryptionkey@0"
I20260811 12:49:48.848389 30982 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 12:49:48.852100 31003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:48.853341 30982 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:49:48.853533 30982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "753adaf7a4a44a71bda2e45051ca1c47"
format_stamp: "Formatted at 2026-08-11 12:49:48 on dist-test-slave-f0tn"
server_key: "c1d9d8724f0b3c6e4fcfa68bc50d10c5"
server_key_iv: "ff723e185c9e5f32353c1193839ba6db"
server_key_version: "encryptionkey@0"
I20260811 12:49:48.853710 30982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:48.871940 30982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:48.875620 30982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:48.875875 30982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:48.893149 30982 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:38457
I20260811 12:49:48.893180 31087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:38457 every 8 connection(s)
I20260811 12:49:48.894505 30982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:49:48.898170 31088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:49:48.898643 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 30982
I20260811 12:49:48.898849 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:49:48.899508 7613 external_mini_cluster.cc:1428] Setting key ebf3f2586521164465e58ca1ef273aef
I20260811 12:49:48.911526 31088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 753adaf7a4a44a71bda2e45051ca1c47: Bootstrap starting.
Aug 11 12:49:48 dist-test-slave-f0tn krb5kdc[30961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:49:48.915400 31088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 753adaf7a4a44a71bda2e45051ca1c47: Neither blocks nor log segments found. Creating new log.
I20260811 12:49:48.916646 31088 log.cc:824] T 00000000000000000000000000000000 P 753adaf7a4a44a71bda2e45051ca1c47: Log is configured to *not* fsync() on all Append() calls
I20260811 12:49:48.919952 31088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 753adaf7a4a44a71bda2e45051ca1c47: No bootstrap required, opened a new log
I20260811 12:49:48.924026 31094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:48.901242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35466 (local address 127.7.111.126:38457)
0811 12:49:48.902070 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:48.902081 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:48.902121 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:49:48.904053 (+ 1932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:48.904077 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:48.904473 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:48.904804 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:48.906654 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:48.907955 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:48.909795 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:48.910431 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:48.910553 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:48.915628 (+ 5075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:48.915636 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:48.915650 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:48.915696 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:48.919286 (+ 3590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:48.920021 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:48.920030 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:48.920038 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:48.920264 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:48.920645 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:48.920650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:48.920653 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:48.921415 (+ 762us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:48.921728 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:48.922156 (+ 428us) server_negotiation.cc:299] Negotiation successful
0811 12:49:48.922617 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":373,"threads_started":1}
I20260811 12:49:48.927929 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:49:48.929904 31088 raft_consensus.cc:348] T 00000000000000000000000000000000 P 753adaf7a4a44a71bda2e45051ca1c47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "753adaf7a4a44a71bda2e45051ca1c47" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38457 } }
I20260811 12:49:48.930271 31088 raft_consensus.cc:374] T 00000000000000000000000000000000 P 753adaf7a4a44a71bda2e45051ca1c47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:49:48.930387 31088 raft_consensus.cc:729] T 00000000000000000000000000000000 P 753adaf7a4a44a71bda2e45051ca1c47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753adaf7a4a44a71bda2e45051ca1c47, State: Initialized, Role: FOLLOWER
I20260811 12:49:48.931015 31088 consensus_queue.cc:260] T 00000000000000000000000000000000 P 753adaf7a4a44a71bda2e45051ca1c47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "753adaf7a4a44a71bda2e45051ca1c47" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38457 } }
I20260811 12:49:48.931286 31088 raft_consensus.cc:388] T 00000000000000000000000000000000 P 753adaf7a4a44a71bda2e45051ca1c47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:49:48.931437 31088 raft_consensus.cc:482] T 00000000000000000000000000000000 P 753adaf7a4a44a71bda2e45051ca1c47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:49:48.931614 31088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 753adaf7a4a44a71bda2e45051ca1c47 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:49:48.933949 31088 raft_consensus.cc:504] T 00000000000000000000000000000000 P 753adaf7a4a44a71bda2e45051ca1c47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "753adaf7a4a44a71bda2e45051ca1c47" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38457 } }
I20260811 12:49:48.934441 31088 leader_election.cc:302] T 00000000000000000000000000000000 P 753adaf7a4a44a71bda2e45051ca1c47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 753adaf7a4a44a71bda2e45051ca1c47; no voters:
I20260811 12:49:48.935030 31088 leader_election.cc:288] T 00000000000000000000000000000000 P 753adaf7a4a44a71bda2e45051ca1c47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:49:48.935141 31096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 753adaf7a4a44a71bda2e45051ca1c47 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:49:48.935842 31096 raft_consensus.cc:686] T 00000000000000000000000000000000 P 753adaf7a4a44a71bda2e45051ca1c47 [term 1 LEADER]: Becoming Leader. State: Replica: 753adaf7a4a44a71bda2e45051ca1c47, State: Running, Role: LEADER
I20260811 12:49:48.936388 31096 consensus_queue.cc:237] T 00000000000000000000000000000000 P 753adaf7a4a44a71bda2e45051ca1c47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "753adaf7a4a44a71bda2e45051ca1c47" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38457 } }
I20260811 12:49:48.937183 31088 sys_catalog.cc:564] T 00000000000000000000000000000000 P 753adaf7a4a44a71bda2e45051ca1c47 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:49:48.938903 31098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 753adaf7a4a44a71bda2e45051ca1c47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 753adaf7a4a44a71bda2e45051ca1c47. Latest consensus state: current_term: 1 leader_uuid: "753adaf7a4a44a71bda2e45051ca1c47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "753adaf7a4a44a71bda2e45051ca1c47" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38457 } } }
I20260811 12:49:48.939380 31098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 753adaf7a4a44a71bda2e45051ca1c47 [sys.catalog]: This master's current role is: LEADER
I20260811 12:49:48.939189 31100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 753adaf7a4a44a71bda2e45051ca1c47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "753adaf7a4a44a71bda2e45051ca1c47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "753adaf7a4a44a71bda2e45051ca1c47" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38457 } } }
I20260811 12:49:48.939584 31100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 753adaf7a4a44a71bda2e45051ca1c47 [sys.catalog]: This master's current role is: LEADER
I20260811 12:49:48.942517 31108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:49:48.947547 31108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:49:48.955722 31108 catalog_manager.cc:1347] Generated new cluster ID: 939878f79e5643f78b1e08ed2f2f60c1
I20260811 12:49:48.955822 31108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:49:49.005511 31108 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:49:49.007532 31108 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:49:49.016129 31108 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 753adaf7a4a44a71bda2e45051ca1c47: Generated new TSK 0
I20260811 12:49:49.017127 31108 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:49:49.041019 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 30982
I20260811 12:49:49.042313 31079 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:49:49.136832 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 30982
2026-08-11T12:49:49Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2243 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:49:49 dist-test-slave-f0tn krb5kdc[31130](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:49:49 dist-test-slave-f0tn krb5kdc[31130](info): set up 2 sockets
Aug 11 12:49:49 dist-test-slave-f0tn krb5kdc[31130](info): commencing operation
krb5kdc: starting...
W20260811 12:49:50.395221 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:49:50 dist-test-slave-f0tn krb5kdc[31130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:49:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:49:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:49:50.702167 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:34145
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:34145
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:49:51.052992 31151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:51.053320 31151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:51.053494 31151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:49:51.061133 31151 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:49:51.061252 31151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:51.061333 31151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:51.061440 31151 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:49:51.061509 31151 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:49:51.072321 31151 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:34145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:34145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:51.073814 31151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:51.075256 31151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:51.087726 31161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:51.087797 31164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:51.087762 31160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:51.088744 31151 server_base.cc:1034] running on GCE node
I20260811 12:49:51.089545 31151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:51.090997 31151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:51.092198 31151 hybrid_clock.cc:648] HybridClock initialized: now 1786452591092137 us; error 51 us; skew 500 ppm
I20260811 12:49:51.092543 31151 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:51 dist-test-slave-f0tn krb5kdc[31130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452591, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:51.095678 31151 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:49:51.097254 31151 webserver.cc:460] Webserver started at http://127.7.111.126:37803/ using document root <none> and password file <none>
I20260811 12:49:51.097942 31151 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:51.098052 31151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:49:51.098369 31151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:49:51.101476 31151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "eb0ec4492a4b4becb717cb1bed99ffaa"
format_stamp: "Formatted at 2026-08-11 12:49:51 on dist-test-slave-f0tn"
server_key: "e047d88286ca8d10ded44e3a1cf68474"
server_key_iv: "a2de3d23a25cf192d6322075bac4483c"
server_key_version: "encryptionkey@0"
I20260811 12:49:51.102268 31151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "eb0ec4492a4b4becb717cb1bed99ffaa"
format_stamp: "Formatted at 2026-08-11 12:49:51 on dist-test-slave-f0tn"
server_key: "e047d88286ca8d10ded44e3a1cf68474"
server_key_iv: "a2de3d23a25cf192d6322075bac4483c"
server_key_version: "encryptionkey@0"
I20260811 12:49:51.107453 31151 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:49:51.110920 31174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:51.112291 31151 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:49:51.112438 31151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "eb0ec4492a4b4becb717cb1bed99ffaa"
format_stamp: "Formatted at 2026-08-11 12:49:51 on dist-test-slave-f0tn"
server_key: "e047d88286ca8d10ded44e3a1cf68474"
server_key_iv: "a2de3d23a25cf192d6322075bac4483c"
server_key_version: "encryptionkey@0"
I20260811 12:49:51.112628 31151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:51.138069 31151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:51.142464 31151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:51.142750 31151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:51.160730 31151 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:34145
I20260811 12:49:51.160787 31255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:34145 every 8 connection(s)
I20260811 12:49:51.162199 31151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:49:51.166164 31256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:49:51.172536 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 31151
I20260811 12:49:51.172737 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:49:51.173365 7613 external_mini_cluster.cc:1428] Setting key ca6df2a8ace0a73af4fe641036dcae5e
I20260811 12:49:51.177995 31256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa: Bootstrap starting.
I20260811 12:49:51.182046 31256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa: Neither blocks nor log segments found. Creating new log.
I20260811 12:49:51.183329 31256 log.cc:824] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa: Log is configured to *not* fsync() on all Append() calls
I20260811 12:49:51.186559 31256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa: No bootstrap required, opened a new log
Aug 11 12:49:51 dist-test-slave-f0tn krb5kdc[31130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:49:51.196681 31256 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34145 } }
I20260811 12:49:51.197230 31256 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:49:51.197361 31256 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0ec4492a4b4becb717cb1bed99ffaa, State: Initialized, Role: FOLLOWER
I20260811 12:49:51.198140 31256 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34145 } }
I20260811 12:49:51.198453 31256 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:49:51.198632 31256 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:49:51.198848 31256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:49:51.198915 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:51.175380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56920 (local address 127.7.111.126:34145)
0811 12:49:51.176225 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:51.176242 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:51.176296 (+ 54us) server_negotiation.cc:407] Connection header received
0811 12:49:51.178698 (+ 2402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:51.178727 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:51.179210 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:51.179654 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:51.180887 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:51.182916 (+ 2029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:51.184507 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:51.185618 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:51.185818 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:51.189702 (+ 3884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:51.189714 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:51.189734 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:51.189797 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:51.194151 (+ 4354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:51.194837 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:51.194847 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:51.194858 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:51.195121 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:51.195473 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:51.195479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:51.195483 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:51.196277 (+ 794us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:51.196603 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:51.196904 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 12:49:51.197425 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":322,"threads_started":1}
I20260811 12:49:51.201432 31256 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34145 } }
I20260811 12:49:51.202044 31256 leader_election.cc:302] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb0ec4492a4b4becb717cb1bed99ffaa; no voters:
I20260811 12:49:51.202616 31256 leader_election.cc:288] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:49:51.202737 31268 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:49:51.203387 31268 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 1 LEADER]: Becoming Leader. State: Replica: eb0ec4492a4b4becb717cb1bed99ffaa, State: Running, Role: LEADER
I20260811 12:49:51.203961 31268 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34145 } }
I20260811 12:49:51.204757 31256 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:49:51.206792 31269 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34145 } } }
I20260811 12:49:51.207126 31269 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [sys.catalog]: This master's current role is: LEADER
I20260811 12:49:51.209511 31270 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb0ec4492a4b4becb717cb1bed99ffaa. Latest consensus state: current_term: 1 leader_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34145 } } }
I20260811 12:49:51.209980 31270 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [sys.catalog]: This master's current role is: LEADER
I20260811 12:49:51.217023 31274 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:49:51.222963 31274 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:49:51.233042 31274 catalog_manager.cc:1347] Generated new cluster ID: d481e234aca34b02b97360605ee49a77
I20260811 12:49:51.233179 31274 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:49:51.250723 31274 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:49:51.252076 31274 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:49:51.275942 31274 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa: Generated new TSK 0
I20260811 12:49:51.276904 31274 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.111.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
I20260811 12:49:51.362104 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34145
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:49:51.709162 31307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:51.709555 31307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:51.709717 31307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:49:51.717571 31307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:51.717703 31307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:51.717852 31307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:49:51.728206 31307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34145
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:51.729663 31307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:51.731135 31307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:51.744534 31316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:51.744565 31314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:51.744581 31321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:51.745167 31307 server_base.cc:1034] running on GCE node
I20260811 12:49:51.745875 31307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:51.746803 31307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:51.748010 31307 hybrid_clock.cc:648] HybridClock initialized: now 1786452591747955 us; error 46 us; skew 500 ppm
I20260811 12:49:51.748327 31307 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:51 dist-test-slave-f0tn krb5kdc[31130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452591, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:51.751482 31307 init.cc:377] Logged in from keytab as kudu/127.7.111.65@KRBTEST.COM (short username kudu)
I20260811 12:49:51.752907 31307 webserver.cc:460] Webserver started at http://127.7.111.65:46329/ using document root <none> and password file <none>
I20260811 12:49:51.753662 31307 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:51.753790 31307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:49:51.754088 31307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:49:51.757037 31307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "4d84abc316b84710a0a524e1e97af4ec"
format_stamp: "Formatted at 2026-08-11 12:49:51 on dist-test-slave-f0tn"
server_key: "2a565d68b86809fb5821b2c5397cba6c"
server_key_iv: "9366a0d3261d03a4309a457e42a960c2"
server_key_version: "encryptionkey@0"
I20260811 12:49:51.757808 31307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "4d84abc316b84710a0a524e1e97af4ec"
format_stamp: "Formatted at 2026-08-11 12:49:51 on dist-test-slave-f0tn"
server_key: "2a565d68b86809fb5821b2c5397cba6c"
server_key_iv: "9366a0d3261d03a4309a457e42a960c2"
server_key_version: "encryptionkey@0"
I20260811 12:49:51.762908 31307 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:49:51.766369 31334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:51.767663 31307 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:49:51.767833 31307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "4d84abc316b84710a0a524e1e97af4ec"
format_stamp: "Formatted at 2026-08-11 12:49:51 on dist-test-slave-f0tn"
server_key: "2a565d68b86809fb5821b2c5397cba6c"
server_key_iv: "9366a0d3261d03a4309a457e42a960c2"
server_key_version: "encryptionkey@0"
I20260811 12:49:51.768018 31307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:51.789211 31307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:51.792929 31307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:51.793190 31307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:51.794173 31307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:49:51.795684 31307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:49:51.795768 31307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:51.795892 31307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:49:51.795945 31307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:51.826756 31307 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:34015
I20260811 12:49:51.826790 31526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:34015 every 8 connection(s)
I20260811 12:49:51.828135 31307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:49:51.832728 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 31307
I20260811 12:49:51.832938 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:49:51.833683 7613 external_mini_cluster.cc:1428] Setting key 007c7742924223d1720b98ef13569046
Aug 11 12:49:51 dist-test-slave-f0tn krb5kdc[31130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452591, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:49:51.847438 31531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:51.830714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:53773 (local address 127.7.111.126:34145)
0811 12:49:51.831173 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:51.831178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:51.832245 (+ 1067us) server_negotiation.cc:407] Connection header received
0811 12:49:51.834138 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:51.834144 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:51.834231 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:51.834329 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:51.836067 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:51.836767 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:51.838319 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:51.838715 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:51.838835 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:51.842138 (+ 3303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:51.842143 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:51.842147 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:51.842179 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:51.844534 (+ 2355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:51.845221 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:51.845226 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:51.845228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:51.845348 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:51.845772 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:51.845775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:51.845778 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:51.846035 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:51.846166 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:51.846838 (+ 672us) server_negotiation.cc:299] Negotiation successful
0811 12:49:51.847060 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":231,"threads_started":1}
I20260811 12:49:51.848347 31532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:51.831324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.65:53773)
0811 12:49:51.832059 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:51.832098 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:51.833876 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:51.834573 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:51.834581 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:51.835278 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:49:51.835904 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:51.835921 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:51.836914 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:51.836918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:51.838171 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:51.838183 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:51.838954 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:51.838962 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:51.839180 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:51.839793 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:49:51.839821 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:49:51.841932 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:49:51.844662 (+ 2730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:51.844671 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:51.844688 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:51.845093 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:51.845456 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:51.845462 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:51.845466 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:51.845656 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:51.846157 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:49:51.846165 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:49:51.846472 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 12:49:51.846699 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 12:49:51.847209 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":286,"threads_started":1}
I20260811 12:49:51.850114 31529 heartbeater.cc:347] Connected to a master server at 127.7.111.126:34145
I20260811 12:49:51.850411 31529 heartbeater.cc:464] Registering TS with master...
I20260811 12:49:51.851127 31529 heartbeater.cc:511] Master 127.7.111.126:34145 requested a full tablet report, sending...
I20260811 12:49:51.853355 31201 ts_manager.cc:194] Registered new tserver with Master: 4d84abc316b84710a0a524e1e97af4ec (127.7.111.65:34015)
I20260811 12:49:51.854844 31201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:53773
WARNING: no policy specified for kudu/127.7.111.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
I20260811 12:49:51.900357 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34145
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:49:52.256793 31536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:52.257103 31536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:52.257265 31536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:49:52.264948 31536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:52.265097 31536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:52.265265 31536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:49:52.275874 31536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34145
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:52.277371 31536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:52.278816 31536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:52.292567 31547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:52.292611 31542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:52.292523 31543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:52.294762 31536 server_base.cc:1034] running on GCE node
I20260811 12:49:52.295440 31536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:52.296437 31536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:52.297660 31536 hybrid_clock.cc:648] HybridClock initialized: now 1786452592297600 us; error 44 us; skew 500 ppm
I20260811 12:49:52.298032 31536 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:52 dist-test-slave-f0tn krb5kdc[31130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452592, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:52.301321 31536 init.cc:377] Logged in from keytab as kudu/127.7.111.66@KRBTEST.COM (short username kudu)
I20260811 12:49:52.302935 31536 webserver.cc:460] Webserver started at http://127.7.111.66:37937/ using document root <none> and password file <none>
I20260811 12:49:52.303673 31536 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:52.303784 31536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:49:52.304080 31536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:49:52.307206 31536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "2bc4568147c94a47862fb557a2abad4e"
format_stamp: "Formatted at 2026-08-11 12:49:52 on dist-test-slave-f0tn"
server_key: "a96dc100b483b944d76ada4098bfa456"
server_key_iv: "970c8d8b5c2ed518a4ba082c3466c681"
server_key_version: "encryptionkey@0"
I20260811 12:49:52.307960 31536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "2bc4568147c94a47862fb557a2abad4e"
format_stamp: "Formatted at 2026-08-11 12:49:52 on dist-test-slave-f0tn"
server_key: "a96dc100b483b944d76ada4098bfa456"
server_key_iv: "970c8d8b5c2ed518a4ba082c3466c681"
server_key_version: "encryptionkey@0"
I20260811 12:49:52.313410 31536 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:49:52.316850 31564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:52.318291 31536 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 12:49:52.318446 31536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "2bc4568147c94a47862fb557a2abad4e"
format_stamp: "Formatted at 2026-08-11 12:49:52 on dist-test-slave-f0tn"
server_key: "a96dc100b483b944d76ada4098bfa456"
server_key_iv: "970c8d8b5c2ed518a4ba082c3466c681"
server_key_version: "encryptionkey@0"
I20260811 12:49:52.318646 31536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:52.335260 31536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:52.339525 31536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:52.339833 31536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:52.340948 31536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:49:52.342424 31536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:49:52.342500 31536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:52.342597 31536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:49:52.342641 31536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:52.374486 31536 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:45189
I20260811 12:49:52.374531 31751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:45189 every 8 connection(s)
I20260811 12:49:52.375761 31536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:49:52.381270 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 31536
I20260811 12:49:52.381470 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:49:52.381909 7613 external_mini_cluster.cc:1428] Setting key 8347eb2a9ea9936efd40f06ab2958e7c
Aug 11 12:49:52 dist-test-slave-f0tn krb5kdc[31130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452592, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:49:52.396113 31756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:52.378296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:33349 (local address 127.7.111.126:34145)
0811 12:49:52.378728 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:52.378732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:52.379625 (+ 893us) server_negotiation.cc:407] Connection header received
0811 12:49:52.381477 (+ 1852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:52.381483 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:52.381573 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:52.381682 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:52.383432 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:52.384322 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:52.385954 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:52.386535 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:52.386660 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:52.390164 (+ 3504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:52.390170 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:52.390176 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:52.390216 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:52.393194 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:52.393890 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:52.393895 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:52.393898 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:52.394044 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:52.394454 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:52.394459 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:52.394463 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:52.394782 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:52.394919 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:52.395566 (+ 647us) server_negotiation.cc:299] Negotiation successful
0811 12:49:52.395778 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":217,"threads_started":1}
I20260811 12:49:52.396888 31757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:52.378727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.66:33349)
0811 12:49:52.379458 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:52.379497 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:52.381195 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:52.381943 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:52.381951 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:52.382496 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:49:52.383252 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:52.383278 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:52.384482 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:52.384487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:52.385814 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:52.385826 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:52.386751 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:52.386756 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:52.386949 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:52.387561 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:49:52.387585 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:49:52.389939 (+ 2354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:49:52.393318 (+ 3379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:52.393324 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:52.393336 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:52.393771 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:52.394161 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:52.394164 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:52.394167 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:52.394348 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:52.394880 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:49:52.394885 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:49:52.395206 (+ 321us) client_negotiation.cc:769] Sending connection context
0811 12:49:52.395435 (+ 229us) client_negotiation.cc:240] Negotiation successful
0811 12:49:52.395800 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":286,"threads_started":1}
I20260811 12:49:52.398412 31752 heartbeater.cc:347] Connected to a master server at 127.7.111.126:34145
I20260811 12:49:52.398725 31752 heartbeater.cc:464] Registering TS with master...
I20260811 12:49:52.399437 31752 heartbeater.cc:511] Master 127.7.111.126:34145 requested a full tablet report, sending...
I20260811 12:49:52.400950 31201 ts_manager.cc:194] Registered new tserver with Master: 2bc4568147c94a47862fb557a2abad4e (127.7.111.66:45189)
I20260811 12:49:52.402050 31201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:33349
WARNING: no policy specified for kudu/127.7.111.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
I20260811 12:49:52.447994 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34145
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:49:52.783778 31761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:52.784051 31761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:52.784179 31761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:49:52.791288 31761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:52.791405 31761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:52.791555 31761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:49:52.801097 31761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:34145
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:52.802417 31761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:52.803720 31761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:52.816820 31768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:52.816746 31770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:52.816794 31773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:52.817860 31761 server_base.cc:1034] running on GCE node
I20260811 12:49:52.818500 31761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:52.819538 31761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:52.820762 31761 hybrid_clock.cc:648] HybridClock initialized: now 1786452592820698 us; error 60 us; skew 500 ppm
I20260811 12:49:52.821130 31761 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:52 dist-test-slave-f0tn krb5kdc[31130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452592, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:52.824493 31761 init.cc:377] Logged in from keytab as kudu/127.7.111.67@KRBTEST.COM (short username kudu)
I20260811 12:49:52.826169 31761 webserver.cc:460] Webserver started at http://127.7.111.67:44699/ using document root <none> and password file <none>
I20260811 12:49:52.827111 31761 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:52.827265 31761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:49:52.827649 31761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:49:52.831760 31761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "1f7d34a9b95647459be576cbf304a48d"
format_stamp: "Formatted at 2026-08-11 12:49:52 on dist-test-slave-f0tn"
server_key: "ac49732928839896ecd9e206fa7ef32c"
server_key_iv: "0b3b2c170fc144658573c8512bf60836"
server_key_version: "encryptionkey@0"
I20260811 12:49:52.832846 31761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "1f7d34a9b95647459be576cbf304a48d"
format_stamp: "Formatted at 2026-08-11 12:49:52 on dist-test-slave-f0tn"
server_key: "ac49732928839896ecd9e206fa7ef32c"
server_key_iv: "0b3b2c170fc144658573c8512bf60836"
server_key_version: "encryptionkey@0"
I20260811 12:49:52.838193 31761 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 12:49:52.841621 31783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:52.842932 31761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:49:52.843124 31761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "1f7d34a9b95647459be576cbf304a48d"
format_stamp: "Formatted at 2026-08-11 12:49:52 on dist-test-slave-f0tn"
server_key: "ac49732928839896ecd9e206fa7ef32c"
server_key_iv: "0b3b2c170fc144658573c8512bf60836"
server_key_version: "encryptionkey@0"
I20260811 12:49:52.843308 31761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:52.858281 31529 heartbeater.cc:503] Master 127.7.111.126:34145 was elected leader, sending a full tablet report...
I20260811 12:49:52.878778 31761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:52.883358 31761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:52.883620 31761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:52.884588 31761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:49:52.885963 31761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:49:52.886054 31761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:52.886155 31761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:49:52.886198 31761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:52.917443 31761 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:43689
I20260811 12:49:52.917505 31978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:43689 every 8 connection(s)
I20260811 12:49:52.918781 31761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:49:52.924189 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 31761
I20260811 12:49:52.924360 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:49:52.924787 7613 external_mini_cluster.cc:1428] Setting key 8663590302a9b2bcc6f3c82cd054d906
Aug 11 12:49:52 dist-test-slave-f0tn krb5kdc[31130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452592, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:49:52.938486 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:52.921214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:52977 (local address 127.7.111.126:34145)
0811 12:49:52.921757 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:52.921764 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:52.922728 (+ 964us) server_negotiation.cc:407] Connection header received
0811 12:49:52.924676 (+ 1948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:52.924681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:52.924780 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:52.924876 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:52.926746 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:52.927884 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:52.929405 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:52.929802 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:52.929911 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:52.933127 (+ 3216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:52.933133 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:52.933137 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:52.933170 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:52.935515 (+ 2345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:52.936186 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:52.936192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:52.936196 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:52.936321 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:52.936731 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:52.936736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:52.936740 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:52.937001 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:52.937145 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:52.937851 (+ 706us) server_negotiation.cc:299] Negotiation successful
0811 12:49:52.938090 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":247,"threads_started":1}
I20260811 12:49:52.939466 31988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:52.921819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.67:52977)
0811 12:49:52.922564 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:52.922602 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:52.924387 (+ 1785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:52.925149 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:52.925157 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:52.925836 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:49:52.926542 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:52.926567 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:52.928036 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:52.928042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:52.929246 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:52.929260 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:52.930027 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:52.930035 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:52.930258 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:52.930820 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:49:52.930847 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:49:52.932926 (+ 2079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:49:52.935644 (+ 2718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:52.935653 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:52.935668 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:52.936066 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:52.936424 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:52.936429 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:52.936433 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:52.936626 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:52.937133 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:49:52.937140 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:49:52.937476 (+ 336us) client_negotiation.cc:769] Sending connection context
0811 12:49:52.937746 (+ 270us) client_negotiation.cc:240] Negotiation successful
0811 12:49:52.938220 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":308,"threads_started":1}
I20260811 12:49:52.940834 31980 heartbeater.cc:347] Connected to a master server at 127.7.111.126:34145
I20260811 12:49:52.941143 31980 heartbeater.cc:464] Registering TS with master...
I20260811 12:49:52.941902 31980 heartbeater.cc:511] Master 127.7.111.126:34145 requested a full tablet report, sending...
I20260811 12:49:52.943234 31201 ts_manager.cc:194] Registered new tserver with Master: 1f7d34a9b95647459be576cbf304a48d (127.7.111.67:43689)
I20260811 12:49:52.943984 31201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:52977
I20260811 12:49:52.951808 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:49:52.967334 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:52.956028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56930 (local address 127.7.111.126:34145)
0811 12:49:52.956174 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:52.956180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:52.956619 (+ 439us) server_negotiation.cc:407] Connection header received
0811 12:49:52.956764 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:52.956768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:52.956835 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:52.956912 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:52.958247 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:52.958945 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:52.960172 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:52.960542 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:52.960625 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:52.962796 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:52.962800 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:52.962803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:52.962831 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:52.965231 (+ 2400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:52.965819 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:52.965822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:52.965824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:52.965934 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:52.966217 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:52.966220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:52.966222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:52.966504 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:52.966689 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:52.966953 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 12:49:52.967120 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 12:49:52.970898 31201 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56930:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:49:52.977680 31201 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:49:53.012598 32012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:52.991635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57096 (local address 127.7.111.67:43689)
0811 12:49:53.003783 (+ 12148us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:53.003793 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:53.004538 (+ 745us) server_negotiation.cc:407] Connection header received
0811 12:49:53.005217 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:53.005225 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:53.005477 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:53.005714 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:49:53.006489 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.007640 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.010164 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.011406 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.011526 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:53.011756 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:53.012067 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 12:49:53.012245 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11960,"thread_start_us":11794,"threads_started":1}
I20260811 12:49:53.012609 32005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:52.991007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:43689 (local address 127.0.0.1:57096)
0811 12:49:53.004420 (+ 13413us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:53.004438 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:53.004605 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:53.005989 (+ 1384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:53.005993 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:53.006039 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:49:53.006331 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:53.006339 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.007786 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.007791 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:53.010014 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:53.010024 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.011580 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.011587 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:53.011782 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:53.011833 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 12:49:53.011964 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 12:49:53.012174 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13294,"mutex_wait_us":1,"thread_start_us":12941,"threads_started":1}
I20260811 12:49:53.013286 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:52.989712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48222 (local address 127.7.111.66:45189)
0811 12:49:53.002968 (+ 13256us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:53.002978 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:53.004360 (+ 1382us) server_negotiation.cc:407] Connection header received
0811 12:49:53.004550 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:53.004559 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:53.004797 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:53.005010 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:49:53.005931 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.007387 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.010117 (+ 2730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.011550 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.011679 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:53.011860 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:53.012572 (+ 712us) server_negotiation.cc:299] Negotiation successful
0811 12:49:53.012973 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13064,"thread_start_us":13262,"threads_started":1}
I20260811 12:49:53.013598 32006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:52.989178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:34015 (local address 127.0.0.1:48302)
0811 12:49:53.000586 (+ 11408us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:53.000625 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:53.000824 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:53.004727 (+ 3903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:53.004734 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:53.004761 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:49:53.005285 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:53.005297 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.007332 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.007338 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:53.009665 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:53.009679 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.012818 (+ 3139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.012826 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:53.012994 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:53.013025 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:49:53.013101 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 12:49:53.013230 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11149,"mutex_wait_us":37,"thread_start_us":15498,"threads_started":1}
I20260811 12:49:53.013739 32009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:52.989536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48302 (local address 127.7.111.65:34015)
0811 12:49:53.004033 (+ 14497us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:53.004041 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:53.004076 (+ 35us) server_negotiation.cc:407] Connection header received
0811 12:49:53.004182 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:53.004192 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:53.004488 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:53.004719 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:49:53.005542 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.007187 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.009815 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.011187 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.011327 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:53.011915 (+ 588us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:53.013249 (+ 1334us) server_negotiation.cc:299] Negotiation successful
0811 12:49:53.013449 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14326,"thread_start_us":14150,"threads_started":1}
I20260811 12:49:53.014123 32011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:52.989325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:45189 (local address 127.0.0.1:48222)
0811 12:49:53.004234 (+ 14909us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:53.004248 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:53.004375 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:53.005257 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:53.005262 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:53.005287 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:49:53.005722 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:53.005735 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.007505 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.007511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:53.009949 (+ 2438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:53.009965 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.012171 (+ 2206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.012179 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:53.012360 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:53.012393 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:49:53.012485 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 12:49:53.012610 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14797,"mutex_wait_us":49,"thread_start_us":10874,"threads_started":1}
I20260811 12:49:53.016937 31865 tablet_service.cc:1467] Processing CreateTablet for tablet 41848ba2a7cc49dfaf4ebaf31258da85 (DEFAULT_TABLE table=test-table [id=4d77cbb229934d95b92e537e59668bec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:49:53.017552 31640 tablet_service.cc:1467] Processing CreateTablet for tablet 41848ba2a7cc49dfaf4ebaf31258da85 (DEFAULT_TABLE table=test-table [id=4d77cbb229934d95b92e537e59668bec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:49:53.017552 31402 tablet_service.cc:1467] Processing CreateTablet for tablet 41848ba2a7cc49dfaf4ebaf31258da85 (DEFAULT_TABLE table=test-table [id=4d77cbb229934d95b92e537e59668bec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:49:53.018378 31865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41848ba2a7cc49dfaf4ebaf31258da85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:49:53.018877 31402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41848ba2a7cc49dfaf4ebaf31258da85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:49:53.018903 31640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41848ba2a7cc49dfaf4ebaf31258da85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:49:53.030229 32013 tablet_bootstrap.cc:492] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d: Bootstrap starting.
I20260811 12:49:53.033061 32017 tablet_bootstrap.cc:492] T 41848ba2a7cc49dfaf4ebaf31258da85 P 2bc4568147c94a47862fb557a2abad4e: Bootstrap starting.
I20260811 12:49:53.033862 32018 tablet_bootstrap.cc:492] T 41848ba2a7cc49dfaf4ebaf31258da85 P 4d84abc316b84710a0a524e1e97af4ec: Bootstrap starting.
I20260811 12:49:53.034200 32013 tablet_bootstrap.cc:654] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d: Neither blocks nor log segments found. Creating new log.
I20260811 12:49:53.035629 32013 log.cc:824] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:49:53.037355 32018 tablet_bootstrap.cc:654] T 41848ba2a7cc49dfaf4ebaf31258da85 P 4d84abc316b84710a0a524e1e97af4ec: Neither blocks nor log segments found. Creating new log.
I20260811 12:49:53.038316 32013 tablet_bootstrap.cc:492] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d: No bootstrap required, opened a new log
I20260811 12:49:53.038575 32013 ts_tablet_manager.cc:1397] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 12:49:53.038667 32018 log.cc:824] T 41848ba2a7cc49dfaf4ebaf31258da85 P 4d84abc316b84710a0a524e1e97af4ec: Log is configured to *not* fsync() on all Append() calls
I20260811 12:49:53.039726 32017 tablet_bootstrap.cc:654] T 41848ba2a7cc49dfaf4ebaf31258da85 P 2bc4568147c94a47862fb557a2abad4e: Neither blocks nor log segments found. Creating new log.
I20260811 12:49:53.040882 32017 log.cc:824] T 41848ba2a7cc49dfaf4ebaf31258da85 P 2bc4568147c94a47862fb557a2abad4e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:49:53.041821 32018 tablet_bootstrap.cc:492] T 41848ba2a7cc49dfaf4ebaf31258da85 P 4d84abc316b84710a0a524e1e97af4ec: No bootstrap required, opened a new log
I20260811 12:49:53.042057 32018 ts_tablet_manager.cc:1397] T 41848ba2a7cc49dfaf4ebaf31258da85 P 4d84abc316b84710a0a524e1e97af4ec: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260811 12:49:53.043897 32017 tablet_bootstrap.cc:492] T 41848ba2a7cc49dfaf4ebaf31258da85 P 2bc4568147c94a47862fb557a2abad4e: No bootstrap required, opened a new log
I20260811 12:49:53.044163 32017 ts_tablet_manager.cc:1397] T 41848ba2a7cc49dfaf4ebaf31258da85 P 2bc4568147c94a47862fb557a2abad4e: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260811 12:49:53.046590 32013 raft_consensus.cc:348] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:49:53.046996 32013 raft_consensus.cc:374] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:49:53.047140 32013 raft_consensus.cc:729] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f7d34a9b95647459be576cbf304a48d, State: Initialized, Role: FOLLOWER
I20260811 12:49:53.047698 32013 consensus_queue.cc:260] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:49:53.048859 31980 heartbeater.cc:503] Master 127.7.111.126:34145 was elected leader, sending a full tablet report...
I20260811 12:49:53.049294 32013 ts_tablet_manager.cc:1428] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d: Time spent starting tablet: real 0.011s user 0.002s sys 0.009s
I20260811 12:49:53.053952 32018 raft_consensus.cc:348] T 41848ba2a7cc49dfaf4ebaf31258da85 P 4d84abc316b84710a0a524e1e97af4ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:49:53.054482 32018 raft_consensus.cc:374] T 41848ba2a7cc49dfaf4ebaf31258da85 P 4d84abc316b84710a0a524e1e97af4ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:49:53.054636 32018 raft_consensus.cc:729] T 41848ba2a7cc49dfaf4ebaf31258da85 P 4d84abc316b84710a0a524e1e97af4ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d84abc316b84710a0a524e1e97af4ec, State: Initialized, Role: FOLLOWER
I20260811 12:49:53.054754 32017 raft_consensus.cc:348] T 41848ba2a7cc49dfaf4ebaf31258da85 P 2bc4568147c94a47862fb557a2abad4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:49:53.055160 32017 raft_consensus.cc:374] T 41848ba2a7cc49dfaf4ebaf31258da85 P 2bc4568147c94a47862fb557a2abad4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:49:53.055285 32017 raft_consensus.cc:729] T 41848ba2a7cc49dfaf4ebaf31258da85 P 2bc4568147c94a47862fb557a2abad4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bc4568147c94a47862fb557a2abad4e, State: Initialized, Role: FOLLOWER
I20260811 12:49:53.055297 32018 consensus_queue.cc:260] T 41848ba2a7cc49dfaf4ebaf31258da85 P 4d84abc316b84710a0a524e1e97af4ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:49:53.055852 32017 consensus_queue.cc:260] T 41848ba2a7cc49dfaf4ebaf31258da85 P 2bc4568147c94a47862fb557a2abad4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:49:53.056979 31752 heartbeater.cc:503] Master 127.7.111.126:34145 was elected leader, sending a full tablet report...
I20260811 12:49:53.057605 32017 ts_tablet_manager.cc:1428] T 41848ba2a7cc49dfaf4ebaf31258da85 P 2bc4568147c94a47862fb557a2abad4e: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260811 12:49:53.057806 32018 ts_tablet_manager.cc:1428] T 41848ba2a7cc49dfaf4ebaf31258da85 P 4d84abc316b84710a0a524e1e97af4ec: Time spent starting tablet: real 0.016s user 0.004s sys 0.010s
I20260811 12:49:53.071165 32025 raft_consensus.cc:482] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:49:53.071408 32025 raft_consensus.cc:504] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:49:53.073030 32025 leader_election.cc:288] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d84abc316b84710a0a524e1e97af4ec (127.7.111.65:34015), 2bc4568147c94a47862fb557a2abad4e (127.7.111.66:45189)
I20260811 12:49:53.079066 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:53.073575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:42081 (local address 127.7.111.66:45189)
0811 12:49:53.073745 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:53.073749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:53.073765 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:49:53.073817 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:53.073822 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:53.073893 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:53.073995 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:49:53.074390 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.075363 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.077172 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.078293 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.078398 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:53.078449 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:53.078795 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 12:49:53.078881 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 12:49:53.079183 31988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:53.073235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:45189 (local address 127.7.111.67:42081)
0811 12:49:53.073446 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:53.073467 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:53.073581 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:53.074038 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:53.074043 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:53.074069 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:49:53.074246 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:53.074257 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.075481 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.075486 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:53.077012 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:53.077024 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.078433 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.078439 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:53.078637 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:53.078672 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 12:49:53.078782 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 12:49:53.078910 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":1}
I20260811 12:49:53.080062 32009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:53.073432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:40259 (local address 127.7.111.65:34015)
0811 12:49:53.073626 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:53.073631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:53.074360 (+ 729us) server_negotiation.cc:407] Connection header received
0811 12:49:53.074501 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:53.074504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:53.074557 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:53.074621 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:49:53.075177 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.076097 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.078076 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.079165 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.079332 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:53.079398 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:53.079807 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 12:49:53.079903 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:49:53.080287 31675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41848ba2a7cc49dfaf4ebaf31258da85" candidate_uuid: "1f7d34a9b95647459be576cbf304a48d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bc4568147c94a47862fb557a2abad4e" is_pre_election: true
W20260811 12:49:53.080441 31530 tablet.cc:2369] T 41848ba2a7cc49dfaf4ebaf31258da85 P 4d84abc316b84710a0a524e1e97af4ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:49:53.080857 31675 raft_consensus.cc:2454] T 41848ba2a7cc49dfaf4ebaf31258da85 P 2bc4568147c94a47862fb557a2abad4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f7d34a9b95647459be576cbf304a48d in term 0.
I20260811 12:49:53.081175 32030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:53.073235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:34015 (local address 127.7.111.67:40259)
0811 12:49:53.074264 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:53.074278 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:53.074395 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:53.074654 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:53.074658 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:53.074684 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:49:53.075054 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:53.075065 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.076243 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.076246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:53.077931 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:53.077946 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.079544 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.079551 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:53.079665 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:53.079687 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 12:49:53.079759 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 12:49:53.079874 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":914,"mutex_wait_us":429,"thread_start_us":271,"threads_started":1}
I20260811 12:49:53.081604 31786 leader_election.cc:302] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f7d34a9b95647459be576cbf304a48d, 2bc4568147c94a47862fb557a2abad4e; no voters:
I20260811 12:49:53.081992 32025 raft_consensus.cc:2781] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:49:53.082151 32025 raft_consensus.cc:482] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:49:53.082257 32025 raft_consensus.cc:3037] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:49:53.082171 31435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41848ba2a7cc49dfaf4ebaf31258da85" candidate_uuid: "1f7d34a9b95647459be576cbf304a48d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d84abc316b84710a0a524e1e97af4ec" is_pre_election: true
I20260811 12:49:53.082670 31435 raft_consensus.cc:2454] T 41848ba2a7cc49dfaf4ebaf31258da85 P 4d84abc316b84710a0a524e1e97af4ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f7d34a9b95647459be576cbf304a48d in term 0.
I20260811 12:49:53.084837 32025 raft_consensus.cc:504] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:49:53.085469 32025 leader_election.cc:288] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d [CANDIDATE]: Term 1 election: Requested vote from peers 4d84abc316b84710a0a524e1e97af4ec (127.7.111.65:34015), 2bc4568147c94a47862fb557a2abad4e (127.7.111.66:45189)
I20260811 12:49:53.085815 31435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41848ba2a7cc49dfaf4ebaf31258da85" candidate_uuid: "1f7d34a9b95647459be576cbf304a48d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d84abc316b84710a0a524e1e97af4ec"
I20260811 12:49:53.086025 31435 raft_consensus.cc:3037] T 41848ba2a7cc49dfaf4ebaf31258da85 P 4d84abc316b84710a0a524e1e97af4ec [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:49:53.085986 31675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41848ba2a7cc49dfaf4ebaf31258da85" candidate_uuid: "1f7d34a9b95647459be576cbf304a48d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bc4568147c94a47862fb557a2abad4e"
I20260811 12:49:53.086197 31675 raft_consensus.cc:3037] T 41848ba2a7cc49dfaf4ebaf31258da85 P 2bc4568147c94a47862fb557a2abad4e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:49:53.089123 31435 raft_consensus.cc:2454] T 41848ba2a7cc49dfaf4ebaf31258da85 P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f7d34a9b95647459be576cbf304a48d in term 1.
I20260811 12:49:53.089349 31675 raft_consensus.cc:2454] T 41848ba2a7cc49dfaf4ebaf31258da85 P 2bc4568147c94a47862fb557a2abad4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f7d34a9b95647459be576cbf304a48d in term 1.
I20260811 12:49:53.089573 31790 leader_election.cc:302] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f7d34a9b95647459be576cbf304a48d, 4d84abc316b84710a0a524e1e97af4ec; no voters:
I20260811 12:49:53.089887 32025 raft_consensus.cc:2781] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:49:53.090325 32025 raft_consensus.cc:686] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d [term 1 LEADER]: Becoming Leader. State: Replica: 1f7d34a9b95647459be576cbf304a48d, State: Running, Role: LEADER
I20260811 12:49:53.090834 32025 consensus_queue.cc:237] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:49:53.095733 31201 catalog_manager.cc:5495] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f7d34a9b95647459be576cbf304a48d (127.7.111.67). New cstate: current_term: 1 leader_uuid: "1f7d34a9b95647459be576cbf304a48d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } health_report { overall_health: HEALTHY } } }
I20260811 12:49:53.113998 32012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:53.108500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57102 (local address 127.7.111.67:43689)
0811 12:49:53.108710 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:53.108717 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:53.108797 (+ 80us) server_negotiation.cc:407] Connection header received
0811 12:49:53.109014 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:53.109018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:53.109095 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:53.109234 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:49:53.109631 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.110365 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.111819 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.112237 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.112347 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:53.112885 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:49:53.113433 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:49:53.113581 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:53.113657 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 12:49:53.113741 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:49:53.122229 31435 raft_consensus.cc:1261] T 41848ba2a7cc49dfaf4ebaf31258da85 P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Refusing update from remote peer 1f7d34a9b95647459be576cbf304a48d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:49:53.122242 31675 raft_consensus.cc:1261] T 41848ba2a7cc49dfaf4ebaf31258da85 P 2bc4568147c94a47862fb557a2abad4e [term 1 FOLLOWER]: Refusing update from remote peer 1f7d34a9b95647459be576cbf304a48d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:49:53.123160 32025 consensus_queue.cc:1035] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:49:53.123450 32031 consensus_queue.cc:1035] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 12:49:53.127782 31754 tablet.cc:2369] T 41848ba2a7cc49dfaf4ebaf31258da85 P 2bc4568147c94a47862fb557a2abad4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:49:53.135279 32036 mvcc.cc:205] Tried to move back new op lower bound from 7317309821418582016 to 7317309821303971840. Current Snapshot: MvccSnapshot[applied={T|T < 7317309821418582016}]
I20260811 12:49:53.136991 32039 mvcc.cc:205] Tried to move back new op lower bound from 7317309821418582016 to 7317309821303971840. Current Snapshot: MvccSnapshot[applied={T|T < 7317309821418582016}]
I20260811 12:49:53.150398 31201 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:49:53.150629 31201 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:49:53.154130 31201 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa: Sending DeleteTablet for 3 replicas of tablet 41848ba2a7cc49dfaf4ebaf31258da85
I20260811 12:49:53.155324 31402 tablet_service.cc:1514] Processing DeleteTablet for tablet 41848ba2a7cc49dfaf4ebaf31258da85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:49:53 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:48302
I20260811 12:49:53.155419 31640 tablet_service.cc:1514] Processing DeleteTablet for tablet 41848ba2a7cc49dfaf4ebaf31258da85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:49:53 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:48222
I20260811 12:49:53.155436 31865 tablet_service.cc:1514] Processing DeleteTablet for tablet 41848ba2a7cc49dfaf4ebaf31258da85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:49:53 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:57096
I20260811 12:49:53.156664 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 31307
I20260811 12:49:53.172134 32055 tablet_replica.cc:330] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d: stopping tablet replica
I20260811 12:49:53.172966 32056 tablet_replica.cc:330] T 41848ba2a7cc49dfaf4ebaf31258da85 P 2bc4568147c94a47862fb557a2abad4e: stopping tablet replica
I20260811 12:49:53.173394 32056 raft_consensus.cc:2229] T 41848ba2a7cc49dfaf4ebaf31258da85 P 2bc4568147c94a47862fb557a2abad4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:49:53.173790 32055 raft_consensus.cc:2229] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:49:53.174108 32056 raft_consensus.cc:2258] T 41848ba2a7cc49dfaf4ebaf31258da85 P 2bc4568147c94a47862fb557a2abad4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:49:53.174532 32055 raft_consensus.cc:2258] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:49:53.176671 32056 ts_tablet_manager.cc:1905] T 41848ba2a7cc49dfaf4ebaf31258da85 P 2bc4568147c94a47862fb557a2abad4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:49:53.176683 32055 ts_tablet_manager.cc:1905] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:49:53.183683 32056 ts_tablet_manager.cc:1918] T 41848ba2a7cc49dfaf4ebaf31258da85 P 2bc4568147c94a47862fb557a2abad4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:49:53.183784 32055 ts_tablet_manager.cc:1918] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:49:53.183939 32056 log.cc:1196] T 41848ba2a7cc49dfaf4ebaf31258da85 P 2bc4568147c94a47862fb557a2abad4e: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/41848ba2a7cc49dfaf4ebaf31258da85
I20260811 12:49:53.184019 32055 log.cc:1196] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/41848ba2a7cc49dfaf4ebaf31258da85
I20260811 12:49:53.184561 32056 ts_tablet_manager.cc:1939] T 41848ba2a7cc49dfaf4ebaf31258da85 P 2bc4568147c94a47862fb557a2abad4e: Deleting consensus metadata
I20260811 12:49:53.184634 32055 ts_tablet_manager.cc:1939] T 41848ba2a7cc49dfaf4ebaf31258da85 P 1f7d34a9b95647459be576cbf304a48d: Deleting consensus metadata
I20260811 12:49:53.186502 31181 catalog_manager.cc:4841] TS 1f7d34a9b95647459be576cbf304a48d (127.7.111.67:43689): tablet 41848ba2a7cc49dfaf4ebaf31258da85 (table test-table [id=4d77cbb229934d95b92e537e59668bec]) successfully deleted
I20260811 12:49:53.186865 31175 catalog_manager.cc:4841] TS 2bc4568147c94a47862fb557a2abad4e (127.7.111.66:45189): tablet 41848ba2a7cc49dfaf4ebaf31258da85 (table test-table [id=4d77cbb229934d95b92e537e59668bec]) successfully deleted
I20260811 12:49:53.194749 32054 tablet_replica.cc:330] T 41848ba2a7cc49dfaf4ebaf31258da85 P 4d84abc316b84710a0a524e1e97af4ec: stopping tablet replica
I20260811 12:49:53.195312 32054 raft_consensus.cc:2229] T 41848ba2a7cc49dfaf4ebaf31258da85 P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:49:53.195940 32054 raft_consensus.cc:2258] T 41848ba2a7cc49dfaf4ebaf31258da85 P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:49:53.197965 32054 ts_tablet_manager.cc:1905] T 41848ba2a7cc49dfaf4ebaf31258da85 P 4d84abc316b84710a0a524e1e97af4ec: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 12:49:53 dist-test-slave-f0tn krb5kdc[31130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.65@KRBTEST.COM
I20260811 12:49:53.204950 32054 ts_tablet_manager.cc:1918] T 41848ba2a7cc49dfaf4ebaf31258da85 P 4d84abc316b84710a0a524e1e97af4ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:49:53.205192 32054 log.cc:1196] T 41848ba2a7cc49dfaf4ebaf31258da85 P 4d84abc316b84710a0a524e1e97af4ec: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/41848ba2a7cc49dfaf4ebaf31258da85
I20260811 12:49:53.205899 32054 ts_tablet_manager.cc:1939] T 41848ba2a7cc49dfaf4ebaf31258da85 P 4d84abc316b84710a0a524e1e97af4ec: Deleting consensus metadata
I20260811 12:49:53.207537 31183 catalog_manager.cc:4841] TS 4d84abc316b84710a0a524e1e97af4ec (127.7.111.65:34015): tablet 41848ba2a7cc49dfaf4ebaf31258da85 (table test-table [id=4d77cbb229934d95b92e537e59668bec]) successfully deleted
I20260811 12:49:53.212741 32009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:53.157896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48308 (local address 127.7.111.65:34015)
0811 12:49:53.158064 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:53.158070 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:53.194010 (+ 35940us) server_negotiation.cc:407] Connection header received
0811 12:49:53.194225 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:53.194231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:53.194337 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:53.194457 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:53.196416 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.197667 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.199538 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.200211 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.200306 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:53.205888 (+ 5582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:53.205907 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:53.205926 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:53.205969 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:53.209771 (+ 3802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:53.210411 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:53.210421 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:53.210425 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:53.210592 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:53.210907 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:53.210911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:53.210915 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:53.211395 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:53.211718 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:53.212123 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 12:49:53.212367 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:49:53.213442 31518 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:49:53.314566 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 31307
I20260811 12:49:53.352377 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 31536
Aug 11 12:49:53 dist-test-slave-f0tn krb5kdc[31130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.66@KRBTEST.COM
I20260811 12:49:53.370223 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:53.353286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48226 (local address 127.7.111.66:45189)
0811 12:49:53.353491 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:53.353495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:53.353530 (+ 35us) server_negotiation.cc:407] Connection header received
0811 12:49:53.353697 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:53.353701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:53.353771 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:53.353861 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:53.355551 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.356294 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.358112 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.358580 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.358711 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:53.364064 (+ 5353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:53.364082 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:53.364099 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:53.364143 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:53.367173 (+ 3030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:53.367990 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:53.368026 (+ 36us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:53.368029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:53.368173 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:53.368566 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:53.368569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:53.368571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:53.368990 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:53.369268 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:53.369663 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 12:49:53.369843 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:49:53.371115 31747 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:49:53.461210 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 31536
I20260811 12:49:53.495868 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 31761
Aug 11 12:49:53 dist-test-slave-f0tn krb5kdc[31130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.67@KRBTEST.COM
I20260811 12:49:53.517053 32012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:53.496788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57110 (local address 127.7.111.67:43689)
0811 12:49:53.496999 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:53.497005 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:53.497031 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:49:53.497171 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:53.497185 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:53.497280 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:53.497464 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:53.498730 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.499974 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.501253 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:53.502131 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:53.502276 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:53.508254 (+ 5978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:53.508274 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:53.508296 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:53.508353 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:53.512706 (+ 4353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:53.513561 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:53.513571 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:53.513575 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:53.513787 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:53.514255 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:53.514261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:53.514264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:53.514767 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:53.515072 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:53.516380 (+ 1308us) server_negotiation.cc:299] Negotiation successful
0811 12:49:53.516612 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 12:49:53.517885 31972 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:49:53.654798 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 31761
I20260811 12:49:53.679899 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 31151
I20260811 12:49:53.681071 31245 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:49:53.791374 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 31151
I20260811 12:49:53.823376 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:34145
--webserver_interface=127.7.111.126
--webserver_port=37803
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:34145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:49:54.155390 32065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:54.155699 32065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:54.155843 32065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:49:54.163044 32065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:49:54.163177 32065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:54.163261 32065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:54.163344 32065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:49:54.163407 32065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:49:54.173285 32065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:34145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:34145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=37803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:54.174577 32065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:54.175909 32065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:54.200660 32072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:54.200595 32075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:54.200603 32073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:54.201504 32065 server_base.cc:1034] running on GCE node
I20260811 12:49:54.202422 32065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:54.204067 32065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:54.205278 32065 hybrid_clock.cc:648] HybridClock initialized: now 1786452594205218 us; error 40 us; skew 500 ppm
I20260811 12:49:54.205726 32065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:54 dist-test-slave-f0tn krb5kdc[31130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452594, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:54.213071 32065 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:49:54.215186 32065 webserver.cc:460] Webserver started at http://127.7.111.126:37803/ using document root <none> and password file <none>
I20260811 12:49:54.216171 32065 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:54.216351 32065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:49:54.223349 32065 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:49:54.226622 32089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:54.228073 32065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:49:54.228262 32065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "eb0ec4492a4b4becb717cb1bed99ffaa"
format_stamp: "Formatted at 2026-08-11 12:49:51 on dist-test-slave-f0tn"
server_key: "e047d88286ca8d10ded44e3a1cf68474"
server_key_iv: "a2de3d23a25cf192d6322075bac4483c"
server_key_version: "encryptionkey@0"
I20260811 12:49:54.229431 32065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:54.250624 32065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:54.255179 32065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:54.255457 32065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:54.273871 32065 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:34145
I20260811 12:49:54.273892 32186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:34145 every 8 connection(s)
I20260811 12:49:54.275452 32065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:49:54.281404 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 32065
I20260811 12:49:54.282269 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:34015
--local_ip_for_outbound_sockets=127.7.111.65
--tserver_master_addrs=127.7.111.126:34145
--webserver_port=46329
--webserver_interface=127.7.111.65
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:49:54.282661 32187 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:49:54.285998 32187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa: Bootstrap starting.
I20260811 12:49:54.309214 32187 log.cc:824] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa: Log is configured to *not* fsync() on all Append() calls
I20260811 12:49:54.325803 32187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:49:54.326416 32187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa: Bootstrap complete.
I20260811 12:49:54.339951 32187 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34145 } }
I20260811 12:49:54.341147 32187 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0ec4492a4b4becb717cb1bed99ffaa, State: Initialized, Role: FOLLOWER
I20260811 12:49:54.341952 32187 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34145 } }
I20260811 12:49:54.342242 32187 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:49:54.342370 32187 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:49:54.342546 32187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 1 FOLLOWER]: Advancing to term 2
I20260811 12:49:54.348176 32187 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34145 } }
I20260811 12:49:54.348803 32187 leader_election.cc:302] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb0ec4492a4b4becb717cb1bed99ffaa; no voters:
I20260811 12:49:54.349458 32187 leader_election.cc:288] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 12:49:54.349622 32191 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 2 FOLLOWER]: Leader election won for term 2
I20260811 12:49:54.350791 32191 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 2 LEADER]: Becoming Leader. State: Replica: eb0ec4492a4b4becb717cb1bed99ffaa, State: Running, Role: LEADER
I20260811 12:49:54.351366 32191 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34145 } }
I20260811 12:49:54.351788 32187 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:49:54.353966 32194 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb0ec4492a4b4becb717cb1bed99ffaa. Latest consensus state: current_term: 2 leader_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34145 } } }
I20260811 12:49:54.354261 32194 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [sys.catalog]: This master's current role is: LEADER
I20260811 12:49:54.353888 32192 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34145 } } }
I20260811 12:49:54.354486 32192 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [sys.catalog]: This master's current role is: LEADER
I20260811 12:49:54.366959 32200 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:49:54.375495 32200 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:49:54.376909 32200 catalog_manager.cc:1259] Loaded cluster ID: d481e234aca34b02b97360605ee49a77
I20260811 12:49:54.377010 32200 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:49:54.380206 32200 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 12:49:54.397742 32200 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 12:49:54.402657 32220 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 12:49:54.407315 32200 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:49:54.409008 32200 catalog_manager.cc:5879] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa: Loaded TSK: 0
I20260811 12:49:54.410533 32200 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 12:49:54.658247 32189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:54.658573 32189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:54.658715 32189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:49:54.666429 32189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:54.666560 32189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:54.666723 32189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:49:54.676914 32189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:34015
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=46329
--tserver_master_addrs=127.7.111.126:34145
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:54.678313 32189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:54.679657 32189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:54.694875 32235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:54.695109 32231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:54.695448 32232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:54.696302 32189 server_base.cc:1034] running on GCE node
I20260811 12:49:54.696951 32189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:54.698030 32189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:54.699255 32189 hybrid_clock.cc:648] HybridClock initialized: now 1786452594699211 us; error 37 us; skew 500 ppm
I20260811 12:49:54.699560 32189 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:54 dist-test-slave-f0tn krb5kdc[31130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452594, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:54.702661 32189 init.cc:377] Logged in from keytab as kudu/127.7.111.65@KRBTEST.COM (short username kudu)
I20260811 12:49:54.704162 32189 webserver.cc:460] Webserver started at http://127.7.111.65:46329/ using document root <none> and password file <none>
I20260811 12:49:54.704854 32189 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:54.704972 32189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:49:54.710364 32189 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:49:54.713037 32246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:54.714424 32189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:49:54.714577 32189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "4d84abc316b84710a0a524e1e97af4ec"
format_stamp: "Formatted at 2026-08-11 12:49:51 on dist-test-slave-f0tn"
server_key: "2a565d68b86809fb5821b2c5397cba6c"
server_key_iv: "9366a0d3261d03a4309a457e42a960c2"
server_key_version: "encryptionkey@0"
I20260811 12:49:54.715518 32189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:54.730248 32189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:54.734251 32189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:54.734514 32189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:54.735436 32189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:49:54.736713 32189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:49:54.736801 32189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:54.736912 32189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:49:54.736992 32189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:54.767360 32189 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:34015
I20260811 12:49:54.767417 32430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:34015 every 8 connection(s)
I20260811 12:49:54.768879 32189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:49:54.779044 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 32189
I20260811 12:49:54.779832 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:45189
--local_ip_for_outbound_sockets=127.7.111.66
--tserver_master_addrs=127.7.111.126:34145
--webserver_port=37937
--webserver_interface=127.7.111.66
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:49:54 dist-test-slave-f0tn krb5kdc[31130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452594, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:49:54.792606 32433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:54.771738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:35931 (local address 127.7.111.126:34145)
0811 12:49:54.772320 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:54.772333 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:54.773000 (+ 667us) server_negotiation.cc:407] Connection header received
0811 12:49:54.775152 (+ 2152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:54.775165 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:54.775470 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:54.775768 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:54.777100 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:54.778355 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:54.780527 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:54.781161 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:54.781303 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:54.785138 (+ 3835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:54.785147 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:54.785162 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:54.785202 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:54.788018 (+ 2816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:54.788771 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:54.788778 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:54.788784 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:54.788955 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:54.789418 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:54.789422 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:54.789424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:54.789924 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:54.790120 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:54.790916 (+ 796us) server_negotiation.cc:299] Negotiation successful
0811 12:49:54.791325 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":236,"threads_started":1}
I20260811 12:49:54.792605 32434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:54.771980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.65:35931)
0811 12:49:54.772849 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:54.772894 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:54.774671 (+ 1777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:54.775886 (+ 1215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:54.775893 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:54.776372 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:49:54.776914 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:54.776931 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:54.778536 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:54.778542 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:54.780396 (+ 1854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:54.780410 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:54.781424 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:54.781431 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:54.781693 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:54.782397 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:49:54.782425 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:49:54.784810 (+ 2385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:49:54.788187 (+ 3377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:54.788197 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:54.788209 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:54.788633 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:54.789071 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:54.789076 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:54.789079 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:54.789284 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:54.790067 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:49:54.790075 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:49:54.790419 (+ 344us) client_negotiation.cc:769] Sending connection context
0811 12:49:54.790666 (+ 247us) client_negotiation.cc:240] Negotiation successful
0811 12:49:54.791255 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":338,"threads_started":1}
I20260811 12:49:54.795877 32431 heartbeater.cc:347] Connected to a master server at 127.7.111.126:34145
I20260811 12:49:54.796216 32431 heartbeater.cc:464] Registering TS with master...
I20260811 12:49:54.797147 32431 heartbeater.cc:511] Master 127.7.111.126:34145 requested a full tablet report, sending...
I20260811 12:49:54.799823 32126 ts_manager.cc:194] Registered new tserver with Master: 4d84abc316b84710a0a524e1e97af4ec (127.7.111.65:34015)
I20260811 12:49:54.801869 32126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:35931
W20260811 12:49:55.127846 32435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:55.128134 32435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:55.128238 32435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:49:55.135279 32435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:55.135381 32435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:55.135500 32435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:49:55.144910 32435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:45189
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=37937
--tserver_master_addrs=127.7.111.126:34145
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:55.146229 32435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:55.147544 32435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:55.160319 32450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:55.160362 32445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:55.160401 32446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:55.161154 32435 server_base.cc:1034] running on GCE node
I20260811 12:49:55.161791 32435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:55.162714 32435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:55.163913 32435 hybrid_clock.cc:648] HybridClock initialized: now 1786452595163862 us; error 44 us; skew 500 ppm
I20260811 12:49:55.164207 32435 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:55 dist-test-slave-f0tn krb5kdc[31130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452595, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:55.167178 32435 init.cc:377] Logged in from keytab as kudu/127.7.111.66@KRBTEST.COM (short username kudu)
I20260811 12:49:55.168531 32435 webserver.cc:460] Webserver started at http://127.7.111.66:37937/ using document root <none> and password file <none>
I20260811 12:49:55.169188 32435 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:55.169301 32435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:49:55.174424 32435 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:49:55.176972 32463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:55.178257 32435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:49:55.178406 32435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "2bc4568147c94a47862fb557a2abad4e"
format_stamp: "Formatted at 2026-08-11 12:49:52 on dist-test-slave-f0tn"
server_key: "a96dc100b483b944d76ada4098bfa456"
server_key_iv: "970c8d8b5c2ed518a4ba082c3466c681"
server_key_version: "encryptionkey@0"
I20260811 12:49:55.179337 32435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:55.212164 32435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:55.216500 32435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:55.216768 32435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:55.217856 32435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:49:55.219681 32435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:49:55.219795 32435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:55.219904 32435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:49:55.219990 32435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:55.250849 32435 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:45189
I20260811 12:49:55.250900 32665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:45189 every 8 connection(s)
I20260811 12:49:55.252337 32435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:49:55.257046 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 32435
I20260811 12:49:55.257773 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:43689
--local_ip_for_outbound_sockets=127.7.111.67
--tserver_master_addrs=127.7.111.126:34145
--webserver_port=44699
--webserver_interface=127.7.111.67
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:49:55 dist-test-slave-f0tn krb5kdc[31130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452595, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:49:55.272377 32433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:55.254892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:45105 (local address 127.7.111.126:34145)
0811 12:49:55.255109 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:55.255115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:55.256281 (+ 1166us) server_negotiation.cc:407] Connection header received
0811 12:49:55.258168 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:55.258174 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:55.258268 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:55.258377 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:55.259803 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.260718 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.262313 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.262903 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.263034 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:55.266433 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:55.266439 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:55.266443 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:55.266476 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:55.269313 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:55.270034 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:55.270040 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:55.270042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:55.270198 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:55.270587 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:55.270592 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:55.270595 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:55.270927 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:55.271063 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:55.271779 (+ 716us) server_negotiation.cc:299] Negotiation successful
0811 12:49:55.272014 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:49:55.273445 32669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:55.255396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.66:45105)
0811 12:49:55.256105 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:55.256143 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:55.257873 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:55.258570 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:55.258578 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:55.259060 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:49:55.259646 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:55.259663 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.260865 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.260869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:55.262141 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:55.262153 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.263093 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.263098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:55.263311 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:55.263942 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:49:55.263965 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:49:55.266222 (+ 2257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:49:55.269484 (+ 3262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:55.269491 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:55.269503 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:55.269912 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:55.270295 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:55.270298 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:55.270300 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:55.270478 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:55.271061 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:49:55.271069 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:49:55.271397 (+ 328us) client_negotiation.cc:769] Sending connection context
0811 12:49:55.271643 (+ 246us) client_negotiation.cc:240] Negotiation successful
0811 12:49:55.272165 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":257,"threads_started":1}
I20260811 12:49:55.274940 32666 heartbeater.cc:347] Connected to a master server at 127.7.111.126:34145
I20260811 12:49:55.275278 32666 heartbeater.cc:464] Registering TS with master...
I20260811 12:49:55.275995 32666 heartbeater.cc:511] Master 127.7.111.126:34145 requested a full tablet report, sending...
I20260811 12:49:55.277475 32126 ts_manager.cc:194] Registered new tserver with Master: 2bc4568147c94a47862fb557a2abad4e (127.7.111.66:45189)
I20260811 12:49:55.278324 32126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:45105
W20260811 12:49:55.615326 32671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:55.615602 32671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:55.615713 32671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:49:55.623054 32671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:55.623165 32671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:55.623296 32671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:49:55.632910 32671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:43689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=44699
--tserver_master_addrs=127.7.111.126:34145
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:55.634270 32671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:55.635628 32671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:55.648854 32680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:55.648910 32682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:55.648886 32679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:55.649959 32671 server_base.cc:1034] running on GCE node
I20260811 12:49:55.650537 32671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:55.651549 32671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:55.652753 32671 hybrid_clock.cc:648] HybridClock initialized: now 1786452595652700 us; error 50 us; skew 500 ppm
I20260811 12:49:55.653055 32671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:55 dist-test-slave-f0tn krb5kdc[31130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452595, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:55.656143 32671 init.cc:377] Logged in from keytab as kudu/127.7.111.67@KRBTEST.COM (short username kudu)
I20260811 12:49:55.657547 32671 webserver.cc:460] Webserver started at http://127.7.111.67:44699/ using document root <none> and password file <none>
I20260811 12:49:55.658211 32671 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:55.658341 32671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:49:55.663964 32671 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.003s
I20260811 12:49:55.666618 32696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:55.667927 32671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:49:55.668077 32671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "1f7d34a9b95647459be576cbf304a48d"
format_stamp: "Formatted at 2026-08-11 12:49:52 on dist-test-slave-f0tn"
server_key: "ac49732928839896ecd9e206fa7ef32c"
server_key_iv: "0b3b2c170fc144658573c8512bf60836"
server_key_version: "encryptionkey@0"
I20260811 12:49:55.669076 32671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:55.688985 32671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:55.692682 32671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:55.692917 32671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:55.693899 32671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:49:55.695178 32671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:49:55.695295 32671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:55.695397 32671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:49:55.695467 32671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:55.725239 32671 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:43689
I20260811 12:49:55.725284 420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:43689 every 8 connection(s)
I20260811 12:49:55.726620 32671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:49:55.732590 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 32671
Aug 11 12:49:55 dist-test-slave-f0tn krb5kdc[31130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452595, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:49:55.746567 32433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:55.729037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:39517 (local address 127.7.111.126:34145)
0811 12:49:55.729213 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:55.729217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:55.730409 (+ 1192us) server_negotiation.cc:407] Connection header received
0811 12:49:55.732152 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:55.732158 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:55.732259 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:55.732375 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:55.734349 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.735143 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.736651 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.737257 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.737368 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:55.740785 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:55.740790 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:55.740793 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:55.740824 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:55.743472 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:55.744173 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:55.744193 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:55.744196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:55.744325 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:55.744806 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:55.744809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:55.744811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:55.745100 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:55.745237 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:55.745994 (+ 757us) server_negotiation.cc:299] Negotiation successful
0811 12:49:55.746227 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55,"spinlock_wait_cycles":4480}
I20260811 12:49:55.747761 426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:55.733515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50200 (local address 127.7.111.126:34145)
0811 12:49:55.734000 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:55.734005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:55.735272 (+ 1267us) server_negotiation.cc:407] Connection header received
0811 12:49:55.735308 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:55.735311 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:55.735366 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:55.735443 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:55.737054 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.737890 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.739310 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.739767 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.739872 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:55.742553 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:55.742562 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:55.742565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:55.742604 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:55.745146 (+ 2542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:55.745842 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:55.745847 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:55.745850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:55.745977 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:55.746344 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:55.746348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:55.746350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:55.746604 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:55.746883 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:55.747309 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 12:49:55.747464 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":229,"threads_started":1}
I20260811 12:49:55.747835 424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:55.729530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.67:39517)
0811 12:49:55.730239 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:55.730286 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:55.731896 (+ 1610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:55.732596 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:55.732604 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:55.733270 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:49:55.734160 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:55.734187 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.735280 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.735284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:55.736509 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:55.736521 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.737519 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.737528 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:55.737768 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:55.738414 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:49:55.738442 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:49:55.740575 (+ 2133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:49:55.743598 (+ 3023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:55.743608 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:55.743624 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:55.744043 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:55.744481 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:55.744485 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:55.744487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:55.744673 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:55.745228 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:49:55.745236 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:49:55.745607 (+ 371us) client_negotiation.cc:769] Sending connection context
0811 12:49:55.745856 (+ 249us) client_negotiation.cc:240] Negotiation successful
0811 12:49:55.746427 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":267,"threads_started":1}
I20260811 12:49:55.749172 421 heartbeater.cc:347] Connected to a master server at 127.7.111.126:34145
I20260811 12:49:55.749531 421 heartbeater.cc:464] Registering TS with master...
I20260811 12:49:55.750365 421 heartbeater.cc:511] Master 127.7.111.126:34145 requested a full tablet report, sending...
I20260811 12:49:55.751704 32126 ts_manager.cc:194] Registered new tserver with Master: 1f7d34a9b95647459be576cbf304a48d (127.7.111.67:43689)
I20260811 12:49:55.752521 32126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:39517
I20260811 12:49:55.762481 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:49:55.769001 426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:55.763316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50212 (local address 127.7.111.126:34145)
0811 12:49:55.763472 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:55.763477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:55.763894 (+ 417us) server_negotiation.cc:407] Connection header received
0811 12:49:55.764071 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:55.764075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:55.764211 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:55.764326 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:49:55.764660 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.765327 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.766781 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.767283 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.767394 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:55.767959 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:49:55.768446 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:49:55.768572 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:55.768686 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 12:49:55.768779 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 12:49:55.769845 32126 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:50212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:49:55.776896 32126 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:49:55.805927 32431 heartbeater.cc:503] Master 127.7.111.126:34145 was elected leader, sending a full tablet report...
I20260811 12:49:55.810235 440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:55.788187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44450 (local address 127.7.111.66:45189)
0811 12:49:55.800697 (+ 12510us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:55.800709 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:55.800746 (+ 37us) server_negotiation.cc:407] Connection header received
0811 12:49:55.800868 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:55.800879 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:55.801170 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:55.802615 (+ 1445us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:49:55.802885 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.804571 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.807481 (+ 2910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.808966 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.809090 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:55.809297 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:55.809751 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 12:49:55.809919 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12286,"thread_start_us":13175,"threads_started":1}
I20260811 12:49:55.810316 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:55.788024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:45189 (local address 127.0.0.1:44450)
0811 12:49:55.797395 (+ 9371us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:55.797446 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:55.797642 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:55.801736 (+ 4094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:55.801760 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:55.801802 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:49:55.802306 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:55.802322 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.804726 (+ 2404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.804731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:55.807293 (+ 2562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:55.807307 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.809226 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.809233 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:55.809507 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:55.809554 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 12:49:55.809677 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 12:49:55.809877 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9045,"mutex_wait_us":25,"thread_start_us":8826,"threads_started":1}
I20260811 12:49:55.813071 443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:55.789116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36214 (local address 127.7.111.67:43689)
0811 12:49:55.803554 (+ 14438us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:55.803564 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:55.805459 (+ 1895us) server_negotiation.cc:407] Connection header received
0811 12:49:55.805576 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:55.805583 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:55.805779 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:55.805979 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:49:55.806792 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.808075 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.810838 (+ 2763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.812021 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.812138 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:55.812380 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:55.812601 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 12:49:55.812765 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14266,"thread_start_us":14110,"threads_started":1}
I20260811 12:49:55.813102 444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:55.788684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:43689 (local address 127.0.0.1:36214)
0811 12:49:55.804003 (+ 15319us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:55.804022 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:55.804156 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:55.806001 (+ 1845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:55.806006 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:55.806032 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:49:55.806549 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:55.806563 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.808216 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.808222 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:55.810009 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:55.810028 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.812233 (+ 2205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.812239 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:55.812418 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:55.812450 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:49:55.812530 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 12:49:55.812656 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15187,"mutex_wait_us":41,"thread_start_us":15056,"threads_started":1}
I20260811 12:49:55.813696 441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:55.788875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:34015 (local address 127.0.0.1:38738)
0811 12:49:55.804351 (+ 15476us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:55.804367 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:55.804468 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:55.806305 (+ 1837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:55.806310 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:55.806336 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:49:55.806841 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:55.806856 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.808987 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.808993 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:55.811019 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:55.811033 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.812892 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.812898 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:55.813069 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:55.813104 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 12:49:55.813188 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 12:49:55.813314 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15366,"mutex_wait_us":1,"thread_start_us":15238,"threads_started":1}
I20260811 12:49:55.814132 445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:55.789282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38738 (local address 127.7.111.65:34015)
0811 12:49:55.804690 (+ 15408us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:55.804697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:55.804723 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:49:55.804816 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:55.804826 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:55.805065 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:55.807064 (+ 1999us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:49:55.807305 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.808846 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.811161 (+ 2315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.812379 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.812510 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:55.812730 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:55.813650 (+ 920us) server_negotiation.cc:299] Negotiation successful
0811 12:49:55.813819 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15251,"thread_start_us":15925,"threads_started":1}
I20260811 12:49:55.814695 32543 tablet_service.cc:1467] Processing CreateTablet for tablet 709ee1329e964c988977413a0baefdda (DEFAULT_TABLE table=test-table [id=9216e3cf92884352b9d93dffb2d5813b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:49:55.816232 32543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 709ee1329e964c988977413a0baefdda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:49:55.816592 311 tablet_service.cc:1467] Processing CreateTablet for tablet 709ee1329e964c988977413a0baefdda (DEFAULT_TABLE table=test-table [id=9216e3cf92884352b9d93dffb2d5813b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:49:55.817791 311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 709ee1329e964c988977413a0baefdda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:49:55.817600 32324 tablet_service.cc:1467] Processing CreateTablet for tablet 709ee1329e964c988977413a0baefdda (DEFAULT_TABLE table=test-table [id=9216e3cf92884352b9d93dffb2d5813b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:49:55.820334 32324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 709ee1329e964c988977413a0baefdda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:49:55.829445 446 tablet_bootstrap.cc:492] T 709ee1329e964c988977413a0baefdda P 1f7d34a9b95647459be576cbf304a48d: Bootstrap starting.
I20260811 12:49:55.831861 448 tablet_bootstrap.cc:492] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e: Bootstrap starting.
I20260811 12:49:55.833117 446 tablet_bootstrap.cc:654] T 709ee1329e964c988977413a0baefdda P 1f7d34a9b95647459be576cbf304a48d: Neither blocks nor log segments found. Creating new log.
I20260811 12:49:55.834600 446 log.cc:824] T 709ee1329e964c988977413a0baefdda P 1f7d34a9b95647459be576cbf304a48d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:49:55.835593 448 tablet_bootstrap.cc:654] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e: Neither blocks nor log segments found. Creating new log.
I20260811 12:49:55.836871 448 log.cc:824] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:49:55.837947 446 tablet_bootstrap.cc:492] T 709ee1329e964c988977413a0baefdda P 1f7d34a9b95647459be576cbf304a48d: No bootstrap required, opened a new log
I20260811 12:49:55.838210 446 ts_tablet_manager.cc:1397] T 709ee1329e964c988977413a0baefdda P 1f7d34a9b95647459be576cbf304a48d: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260811 12:49:55.839681 448 tablet_bootstrap.cc:492] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e: No bootstrap required, opened a new log
I20260811 12:49:55.839918 448 ts_tablet_manager.cc:1397] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 12:49:55.845752 453 tablet_bootstrap.cc:492] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec: Bootstrap starting.
I20260811 12:49:55.846048 446 raft_consensus.cc:348] T 709ee1329e964c988977413a0baefdda P 1f7d34a9b95647459be576cbf304a48d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:49:55.846506 446 raft_consensus.cc:374] T 709ee1329e964c988977413a0baefdda P 1f7d34a9b95647459be576cbf304a48d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:49:55.846616 446 raft_consensus.cc:729] T 709ee1329e964c988977413a0baefdda P 1f7d34a9b95647459be576cbf304a48d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f7d34a9b95647459be576cbf304a48d, State: Initialized, Role: FOLLOWER
I20260811 12:49:55.847169 448 raft_consensus.cc:348] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:49:55.847277 446 consensus_queue.cc:260] T 709ee1329e964c988977413a0baefdda P 1f7d34a9b95647459be576cbf304a48d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:49:55.847479 448 raft_consensus.cc:374] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:49:55.847580 448 raft_consensus.cc:729] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bc4568147c94a47862fb557a2abad4e, State: Initialized, Role: FOLLOWER
I20260811 12:49:55.848222 448 consensus_queue.cc:260] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:49:55.848459 421 heartbeater.cc:503] Master 127.7.111.126:34145 was elected leader, sending a full tablet report...
I20260811 12:49:55.849076 446 ts_tablet_manager.cc:1428] T 709ee1329e964c988977413a0baefdda P 1f7d34a9b95647459be576cbf304a48d: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260811 12:49:55.849398 453 tablet_bootstrap.cc:654] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec: Neither blocks nor log segments found. Creating new log.
I20260811 12:49:55.850135 32666 heartbeater.cc:503] Master 127.7.111.126:34145 was elected leader, sending a full tablet report...
I20260811 12:49:55.850574 453 log.cc:824] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec: Log is configured to *not* fsync() on all Append() calls
I20260811 12:49:55.852121 448 ts_tablet_manager.cc:1428] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e: Time spent starting tablet: real 0.012s user 0.001s sys 0.011s
I20260811 12:49:55.854851 453 tablet_bootstrap.cc:492] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec: No bootstrap required, opened a new log
I20260811 12:49:55.855149 453 ts_tablet_manager.cc:1397] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.006s
I20260811 12:49:55.862172 453 raft_consensus.cc:348] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:49:55.862530 453 raft_consensus.cc:374] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:49:55.862638 453 raft_consensus.cc:729] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d84abc316b84710a0a524e1e97af4ec, State: Initialized, Role: FOLLOWER
I20260811 12:49:55.863162 453 consensus_queue.cc:260] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:49:55.864509 453 ts_tablet_manager.cc:1428] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec: Time spent starting tablet: real 0.009s user 0.006s sys 0.003s
I20260811 12:49:55.938200 456 raft_consensus.cc:482] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:49:55.938491 456 raft_consensus.cc:504] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:49:55.940127 456 leader_election.cc:288] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d84abc316b84710a0a524e1e97af4ec (127.7.111.65:34015), 1f7d34a9b95647459be576cbf304a48d (127.7.111.67:43689)
I20260811 12:49:55.947300 445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:55.940448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:46175 (local address 127.7.111.65:34015)
0811 12:49:55.940659 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:55.940665 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:55.941670 (+ 1005us) server_negotiation.cc:407] Connection header received
0811 12:49:55.942089 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:55.942095 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:55.942190 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:55.942287 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:49:55.942934 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.943701 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.945575 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.946351 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.946474 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:55.946545 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:55.946928 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 12:49:55.947050 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:49:55.947314 443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:55.940788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:40181 (local address 127.7.111.67:43689)
0811 12:49:55.940963 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:55.940968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:55.941939 (+ 971us) server_negotiation.cc:407] Connection header received
0811 12:49:55.942209 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:55.942216 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:55.942296 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:55.942379 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:49:55.943095 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.943816 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.945638 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.946411 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.946589 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:55.946659 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:55.946951 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 12:49:55.947073 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:49:55.947422 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:55.940320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:34015 (local address 127.7.111.66:46175)
0811 12:49:55.941543 (+ 1223us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:55.941566 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:55.941810 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:55.942360 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:55.942366 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:55.942391 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:49:55.942775 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:55.942788 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.943840 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.943846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:55.945407 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:55.945422 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.946589 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.946596 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:55.946771 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:55.946804 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:49:55.946875 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 12:49:55.946988 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1078,"mutex_wait_us":24,"thread_start_us":838,"threads_started":1}
I20260811 12:49:55.947628 467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:55.940570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:43689 (local address 127.7.111.66:40181)
0811 12:49:55.941842 (+ 1272us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:55.941864 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:55.942036 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:55.942500 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:55.942505 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:55.942548 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:49:55.942930 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:55.942941 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.943955 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.943960 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:55.945485 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:55.945502 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.946647 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.946652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:55.946801 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:55.946823 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 12:49:55.946907 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 12:49:55.947027 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1137,"mutex_wait_us":26,"thread_start_us":914,"threads_started":1}
I20260811 12:49:55.947917 458 raft_consensus.cc:482] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:49:55.948233 458 raft_consensus.cc:504] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:49:55.948450 32352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "709ee1329e964c988977413a0baefdda" candidate_uuid: "2bc4568147c94a47862fb557a2abad4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d84abc316b84710a0a524e1e97af4ec" is_pre_election: true
I20260811 12:49:55.949031 32352 raft_consensus.cc:2454] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bc4568147c94a47862fb557a2abad4e in term 0.
I20260811 12:49:55.949335 337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "709ee1329e964c988977413a0baefdda" candidate_uuid: "2bc4568147c94a47862fb557a2abad4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f7d34a9b95647459be576cbf304a48d" is_pre_election: true
I20260811 12:49:55.949769 32468 leader_election.cc:302] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bc4568147c94a47862fb557a2abad4e, 4d84abc316b84710a0a524e1e97af4ec; no voters:
I20260811 12:49:55.949862 337 raft_consensus.cc:2454] T 709ee1329e964c988977413a0baefdda P 1f7d34a9b95647459be576cbf304a48d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bc4568147c94a47862fb557a2abad4e in term 0.
I20260811 12:49:55.950217 458 leader_election.cc:288] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2bc4568147c94a47862fb557a2abad4e (127.7.111.66:45189), 1f7d34a9b95647459be576cbf304a48d (127.7.111.67:43689)
I20260811 12:49:55.951294 456 raft_consensus.cc:2781] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:49:55.951450 456 raft_consensus.cc:482] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:49:55.951551 456 raft_consensus.cc:3037] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:49:55.954985 456 raft_consensus.cc:504] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:49:55.956108 32352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "709ee1329e964c988977413a0baefdda" candidate_uuid: "2bc4568147c94a47862fb557a2abad4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d84abc316b84710a0a524e1e97af4ec"
I20260811 12:49:55.956435 32352 raft_consensus.cc:3037] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:49:55.956539 456 leader_election.cc:288] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e [CANDIDATE]: Term 1 election: Requested vote from peers 4d84abc316b84710a0a524e1e97af4ec (127.7.111.65:34015), 1f7d34a9b95647459be576cbf304a48d (127.7.111.67:43689)
I20260811 12:49:55.956681 337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "709ee1329e964c988977413a0baefdda" candidate_uuid: "2bc4568147c94a47862fb557a2abad4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f7d34a9b95647459be576cbf304a48d"
I20260811 12:49:55.956898 337 raft_consensus.cc:3037] T 709ee1329e964c988977413a0baefdda P 1f7d34a9b95647459be576cbf304a48d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:49:55.960311 337 raft_consensus.cc:2454] T 709ee1329e964c988977413a0baefdda P 1f7d34a9b95647459be576cbf304a48d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bc4568147c94a47862fb557a2abad4e in term 1.
I20260811 12:49:55.960361 32352 raft_consensus.cc:2454] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bc4568147c94a47862fb557a2abad4e in term 1.
I20260811 12:49:55.960799 32467 leader_election.cc:302] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f7d34a9b95647459be576cbf304a48d, 2bc4568147c94a47862fb557a2abad4e; no voters:
I20260811 12:49:55.961300 456 raft_consensus.cc:2781] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:49:55.961854 456 raft_consensus.cc:686] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e [term 1 LEADER]: Becoming Leader. State: Replica: 2bc4568147c94a47862fb557a2abad4e, State: Running, Role: LEADER
I20260811 12:49:55.962455 456 consensus_queue.cc:237] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:49:55.967620 32123 catalog_manager.cc:5495] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e reported cstate change: term changed from 0 to 1, leader changed from <none> to 2bc4568147c94a47862fb557a2abad4e (127.7.111.66). New cstate: current_term: 1 leader_uuid: "2bc4568147c94a47862fb557a2abad4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } health_report { overall_health: UNKNOWN } } }
I20260811 12:49:55.969970 443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:55.950640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:51727 (local address 127.7.111.67:43689)
0811 12:49:55.950804 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:55.950809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:55.962828 (+ 12019us) server_negotiation.cc:407] Connection header received
0811 12:49:55.962918 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:55.962925 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:55.963016 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:55.963221 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:49:55.963851 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.965125 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.967922 (+ 2797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.968912 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.969134 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:55.969215 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:55.969591 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 12:49:55.969716 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 12:49:55.970124 469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:55.950485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:43689 (local address 127.7.111.65:51727)
0811 12:49:55.962633 (+ 12148us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:55.962651 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:55.962789 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:55.963270 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:55.963275 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:55.963313 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:49:55.963697 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:55.963711 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.965319 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.965326 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:55.967702 (+ 2376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:55.967716 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.969215 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.969222 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:55.969421 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:55.969454 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:49:55.969567 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 12:49:55.969707 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12020,"mutex_wait_us":1,"thread_start_us":16075,"threads_started":1}
I20260811 12:49:55.971544 337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "709ee1329e964c988977413a0baefdda" candidate_uuid: "4d84abc316b84710a0a524e1e97af4ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f7d34a9b95647459be576cbf304a48d" is_pre_election: true
I20260811 12:49:55.971815 337 raft_consensus.cc:2379] T 709ee1329e964c988977413a0baefdda P 1f7d34a9b95647459be576cbf304a48d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4d84abc316b84710a0a524e1e97af4ec in current term 1: Already voted for candidate 2bc4568147c94a47862fb557a2abad4e in this term.
I20260811 12:49:55.973588 440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:55.950568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:50247 (local address 127.7.111.66:45189)
0811 12:49:55.950887 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:55.950892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:55.966536 (+ 15644us) server_negotiation.cc:407] Connection header received
0811 12:49:55.966952 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:55.966958 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:55.967024 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:55.967114 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:49:55.967869 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.968861 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.971270 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.972440 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.972564 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:55.972640 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:55.973264 (+ 624us) server_negotiation.cc:299] Negotiation successful
0811 12:49:55.973360 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205}
I20260811 12:49:55.973595 468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:55.950516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:45189 (local address 127.7.111.65:50247)
0811 12:49:55.966407 (+ 15891us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:55.966428 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:55.966826 (+ 398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:55.967151 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:55.967156 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:55.967183 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:49:55.967701 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:55.967716 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.969074 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.969080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:55.971067 (+ 1987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:55.971084 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.972876 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.972884 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:55.973073 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:55.973112 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 12:49:55.973200 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 12:49:55.973295 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15759,"thread_start_us":12330,"threads_started":1}
I20260811 12:49:55.974435 32585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "709ee1329e964c988977413a0baefdda" candidate_uuid: "4d84abc316b84710a0a524e1e97af4ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bc4568147c94a47862fb557a2abad4e" is_pre_election: true
I20260811 12:49:55.975347 32247 leader_election.cc:302] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4d84abc316b84710a0a524e1e97af4ec; no voters: 1f7d34a9b95647459be576cbf304a48d, 2bc4568147c94a47862fb557a2abad4e
I20260811 12:49:55.975705 458 raft_consensus.cc:2726] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 12:49:55.978088 422 tablet.cc:2369] T 709ee1329e964c988977413a0baefdda P 1f7d34a9b95647459be576cbf304a48d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:49:55.991082 440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:55.986074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44466 (local address 127.7.111.66:45189)
0811 12:49:55.986256 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:55.986262 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:55.986416 (+ 154us) server_negotiation.cc:407] Connection header received
0811 12:49:55.986584 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:55.986588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:55.986659 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:55.986772 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:49:55.987060 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.987748 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.989142 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:55.989572 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:55.989659 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:55.990125 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:49:55.990584 (+ 459us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:49:55.990689 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:55.990749 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 12:49:55.990809 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:49:55.998649 337 raft_consensus.cc:1261] T 709ee1329e964c988977413a0baefdda P 1f7d34a9b95647459be576cbf304a48d [term 1 FOLLOWER]: Refusing update from remote peer 2bc4568147c94a47862fb557a2abad4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:49:55.998819 32352 raft_consensus.cc:1261] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Refusing update from remote peer 2bc4568147c94a47862fb557a2abad4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:49:55.999469 470 consensus_queue.cc:1035] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:49:55.999866 456 consensus_queue.cc:1035] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 12:49:56.004283 32667 tablet.cc:2369] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:49:56.025579 32120 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:50212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:49:56.025893 32120 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:50212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:49:56.029531 32120 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa: Sending DeleteTablet for 3 replicas of tablet 709ee1329e964c988977413a0baefdda
I20260811 12:49:56.030653 32543 tablet_service.cc:1514] Processing DeleteTablet for tablet 709ee1329e964c988977413a0baefdda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:49:56 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:44450
I20260811 12:49:56.030681 311 tablet_service.cc:1514] Processing DeleteTablet for tablet 709ee1329e964c988977413a0baefdda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:49:56 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:36214
I20260811 12:49:56.030694 32324 tablet_service.cc:1514] Processing DeleteTablet for tablet 709ee1329e964c988977413a0baefdda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:49:56 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:38738
I20260811 12:49:56.032433 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 32189
I20260811 12:49:56.050058 445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:56.033473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38752 (local address 127.7.111.65:34015)
0811 12:49:56.033635 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:56.033641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:56.033661 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:49:56.033759 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:56.033764 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:56.033839 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:56.033938 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:56.035709 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:56.036879 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:56.038844 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:56.039471 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:56.039591 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:56.043034 (+ 3443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:56.043048 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:56.043066 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:56.043111 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:56.046866 (+ 3755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:56.047778 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:56.047788 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:56.047792 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:56.047959 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:56.048331 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:56.048336 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:56.048339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:56.048803 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:56.049090 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:56.049434 (+ 344us) server_negotiation.cc:299] Negotiation successful
0811 12:49:56.049657 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:49:56.050756 32426 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:49:56.055099 490 tablet_replica.cc:330] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e: stopping tablet replica
I20260811 12:49:56.055783 489 tablet_replica.cc:330] T 709ee1329e964c988977413a0baefdda P 1f7d34a9b95647459be576cbf304a48d: stopping tablet replica
I20260811 12:49:56.056210 489 raft_consensus.cc:2229] T 709ee1329e964c988977413a0baefdda P 1f7d34a9b95647459be576cbf304a48d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:49:56.056684 489 raft_consensus.cc:2258] T 709ee1329e964c988977413a0baefdda P 1f7d34a9b95647459be576cbf304a48d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:49:56.057401 490 raft_consensus.cc:2229] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:49:56.058203 490 raft_consensus.cc:2258] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:49:56.059247 489 ts_tablet_manager.cc:1905] T 709ee1329e964c988977413a0baefdda P 1f7d34a9b95647459be576cbf304a48d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:49:56.060418 490 ts_tablet_manager.cc:1905] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:49:56.066567 489 ts_tablet_manager.cc:1918] T 709ee1329e964c988977413a0baefdda P 1f7d34a9b95647459be576cbf304a48d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:49:56.066810 489 log.cc:1196] T 709ee1329e964c988977413a0baefdda P 1f7d34a9b95647459be576cbf304a48d: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/709ee1329e964c988977413a0baefdda
I20260811 12:49:56.067420 489 ts_tablet_manager.cc:1939] T 709ee1329e964c988977413a0baefdda P 1f7d34a9b95647459be576cbf304a48d: Deleting consensus metadata
I20260811 12:49:56.068357 490 ts_tablet_manager.cc:1918] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:49:56.068611 490 log.cc:1196] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/709ee1329e964c988977413a0baefdda
I20260811 12:49:56.069274 490 ts_tablet_manager.cc:1939] T 709ee1329e964c988977413a0baefdda P 2bc4568147c94a47862fb557a2abad4e: Deleting consensus metadata
I20260811 12:49:56.069484 32095 catalog_manager.cc:4841] TS 1f7d34a9b95647459be576cbf304a48d (127.7.111.67:43689): tablet 709ee1329e964c988977413a0baefdda (table test-table [id=9216e3cf92884352b9d93dffb2d5813b]) successfully deleted
I20260811 12:49:56.070890 32091 catalog_manager.cc:4841] TS 2bc4568147c94a47862fb557a2abad4e (127.7.111.66:45189): tablet 709ee1329e964c988977413a0baefdda (table test-table [id=9216e3cf92884352b9d93dffb2d5813b]) successfully deleted
I20260811 12:49:56.174643 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 32189
I20260811 12:49:56.174803 491 tablet_replica.cc:330] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec: stopping tablet replica
W20260811 12:49:56.199690 32096 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.111.65:34015 (error 108)
I20260811 12:49:56.199928 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 32435
I20260811 12:49:56.201768 441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:56.201281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:34015 (local address 127.0.0.1:38768)
0811 12:49:56.201479 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:56.201601 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:34015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 12:49:56.202004 32096 catalog_manager.cc:4584] TS 4d84abc316b84710a0a524e1e97af4ec (127.7.111.65:34015): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 709ee1329e964c988977413a0baefdda: Network error: Client connection negotiation failed: client connection to 127.7.111.65:34015: connect: Connection refused (error 111)
I20260811 12:49:56.215972 440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:56.200973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44468 (local address 127.7.111.66:45189)
0811 12:49:56.201205 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:56.201211 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:56.201227 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:49:56.201279 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:56.201283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:56.201359 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:56.201491 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:56.203311 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:56.204048 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:56.206070 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:56.206484 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:56.206579 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:56.209927 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:56.209940 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:56.209958 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:56.209997 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:56.213017 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:56.213915 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:56.213923 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:56.213925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:56.214077 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:56.214497 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:56.214500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:56.214502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:56.214921 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:56.215151 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:56.215509 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 12:49:56.215686 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:49:56.216729 32660 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:49:56.255740 441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:56.255273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:34015 (local address 127.0.0.1:38772)
0811 12:49:56.255466 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:56.255564 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:34015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 12:49:56.256718 441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:56.256389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:34015 (local address 127.0.0.1:38776)
0811 12:49:56.256506 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:56.256591 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:34015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 12:49:56.317152 441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:56.316711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:34015 (local address 127.0.0.1:38778)
0811 12:49:56.316903 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:56.317003 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:34015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 12:49:56.318255 441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:56.317936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:34015 (local address 127.0.0.1:38792)
0811 12:49:56.318062 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:56.318140 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:34015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 12:49:56.340584 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 32435
I20260811 12:49:56.367342 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 32671
I20260811 12:49:56.380122 443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:56.368088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36224 (local address 127.7.111.67:43689)
0811 12:49:56.368277 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:56.368281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:56.368332 (+ 51us) server_negotiation.cc:407] Connection header received
0811 12:49:56.368518 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:56.368522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:56.368605 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:56.368689 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:56.370092 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:56.370764 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:56.371968 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:56.372382 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:56.372465 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:56.375039 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:56.375057 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:56.375075 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:56.375111 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:56.377844 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:56.378461 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:56.378468 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:56.378471 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:56.378589 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:56.378862 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:56.378865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:56.378867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:56.379205 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:56.379418 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:56.379658 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 12:49:56.379827 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:49:56.380738 410 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:49:56.398976 441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:56.398492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:34015 (local address 127.0.0.1:38808)
0811 12:49:56.398704 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:56.398797 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:34015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 12:49:56.400089 441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:56.399712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:34015 (local address 127.0.0.1:38824)
0811 12:49:56.399906 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:56.399979 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:34015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 12:49:56.468321 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 32671
I20260811 12:49:56.494710 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 32065
I20260811 12:49:56.495882 32179 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:49:56.612522 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 32065
I20260811 12:49:56.612550 441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:56.612042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:34015 (local address 127.0.0.1:38838)
0811 12:49:56.612281 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:56.612376 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:34015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 12:49:56.644487 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:34145
--webserver_interface=127.7.111.126
--webserver_port=37803
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:34145
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:49:56.980187 506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:56.980489 506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:56.980630 506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:49:56.987741 506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:49:56.987846 506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:56.987918 506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:56.987998 506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:49:56.988056 506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:49:56.997813 506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:34145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:34145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=37803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:56.999096 506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:57.000419 506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:57.011804 514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:57.011888 521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:57.011823 515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:57.012914 506 server_base.cc:1034] running on GCE node
I20260811 12:49:57.013742 506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:57.015074 506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:57.016284 506 hybrid_clock.cc:648] HybridClock initialized: now 1786452597016228 us; error 42 us; skew 500 ppm
I20260811 12:49:57.016638 506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:57 dist-test-slave-f0tn krb5kdc[31130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452597, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:57.019644 506 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:49:57.021028 506 webserver.cc:460] Webserver started at http://127.7.111.126:37803/ using document root <none> and password file <none>
I20260811 12:49:57.021726 506 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:57.021859 506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:49:57.027005 506 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 12:49:57.029676 538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:57.030891 506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:49:57.031047 506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "eb0ec4492a4b4becb717cb1bed99ffaa"
format_stamp: "Formatted at 2026-08-11 12:49:51 on dist-test-slave-f0tn"
server_key: "e047d88286ca8d10ded44e3a1cf68474"
server_key_iv: "a2de3d23a25cf192d6322075bac4483c"
server_key_version: "encryptionkey@0"
I20260811 12:49:57.031965 506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:57.052471 506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:57.056365 506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:57.056617 506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:57.073767 506 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:34145
I20260811 12:49:57.073799 632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:34145 every 8 connection(s)
I20260811 12:49:57.075183 506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:49:57.081977 633 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:49:57.082214 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 506
I20260811 12:49:57.083016 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:34015
--local_ip_for_outbound_sockets=127.7.111.65
--tserver_master_addrs=127.7.111.126:34145
--webserver_port=46329
--webserver_interface=127.7.111.65
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:49:57.085198 633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa: Bootstrap starting.
I20260811 12:49:57.106490 633 log.cc:824] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa: Log is configured to *not* fsync() on all Append() calls
I20260811 12:49:57.122565 633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:49:57.123070 633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa: Bootstrap complete.
I20260811 12:49:57.132128 633 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34145 } }
I20260811 12:49:57.132954 633 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0ec4492a4b4becb717cb1bed99ffaa, State: Initialized, Role: FOLLOWER
I20260811 12:49:57.133564 633 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34145 } }
I20260811 12:49:57.133783 633 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:49:57.133900 633 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:49:57.134052 633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 2 FOLLOWER]: Advancing to term 3
I20260811 12:49:57.137940 633 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34145 } }
I20260811 12:49:57.138409 633 leader_election.cc:302] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb0ec4492a4b4becb717cb1bed99ffaa; no voters:
I20260811 12:49:57.139040 633 leader_election.cc:288] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 12:49:57.139180 643 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 3 FOLLOWER]: Leader election won for term 3
I20260811 12:49:57.140311 643 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [term 3 LEADER]: Becoming Leader. State: Replica: eb0ec4492a4b4becb717cb1bed99ffaa, State: Running, Role: LEADER
I20260811 12:49:57.140888 643 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34145 } }
I20260811 12:49:57.141166 633 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:49:57.143402 646 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb0ec4492a4b4becb717cb1bed99ffaa. Latest consensus state: current_term: 3 leader_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34145 } } }
I20260811 12:49:57.143805 646 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [sys.catalog]: This master's current role is: LEADER
I20260811 12:49:57.144383 644 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0ec4492a4b4becb717cb1bed99ffaa" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 34145 } } }
I20260811 12:49:57.144687 644 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0ec4492a4b4becb717cb1bed99ffaa [sys.catalog]: This master's current role is: LEADER
I20260811 12:49:57.149729 658 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:49:57.158169 658 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:49:57.159649 658 catalog_manager.cc:1259] Loaded cluster ID: d481e234aca34b02b97360605ee49a77
I20260811 12:49:57.159790 658 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 12:49:57.161823 658 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f091afbc9ad google::LogMessage::Fail() at ??:0
@ 0x7f091afbf88a google::LogMessage::SendToLog() at ??:0
@ 0x7f091afbc4de google::LogMessage::Flush() at ??:0
@ 0x7f091afbda19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f09309f3512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f09309f0976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0930a78610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f0930a78439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f091dc08345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f091dc0b91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f091dc0b799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f091dbe6eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0920ec06db start_thread at ??:0
@ 0x7f0919e5661f clone at ??:0
W20260811 12:49:57.478847 638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:57.479157 638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:57.479303 638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:49:57.486505 638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:57.486608 638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:57.486759 638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:49:57.496230 638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:34015
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=46329
--tserver_master_addrs=127.7.111.126:34145
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:57.497522 638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:57.498867 638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:57.511524 713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:57.511564 707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:57.511605 705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:57.517668 638 server_base.cc:1034] running on GCE node
I20260811 12:49:57.518229 638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:57.519104 638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:57.520303 638 hybrid_clock.cc:648] HybridClock initialized: now 1786452597520261 us; error 40 us; skew 500 ppm
I20260811 12:49:57.520602 638 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:57 dist-test-slave-f0tn krb5kdc[31130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452597, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:57.523564 638 init.cc:377] Logged in from keytab as kudu/127.7.111.65@KRBTEST.COM (short username kudu)
I20260811 12:49:57.524979 638 webserver.cc:460] Webserver started at http://127.7.111.65:46329/ using document root <none> and password file <none>
I20260811 12:49:57.525717 638 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:57.525835 638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:49:57.530922 638 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:49:57.535692 722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:57.536877 638 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 12:49:57.537025 638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "4d84abc316b84710a0a524e1e97af4ec"
format_stamp: "Formatted at 2026-08-11 12:49:51 on dist-test-slave-f0tn"
server_key: "2a565d68b86809fb5821b2c5397cba6c"
server_key_iv: "9366a0d3261d03a4309a457e42a960c2"
server_key_version: "encryptionkey@0"
I20260811 12:49:57.537995 638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:57.567098 638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:57.571059 638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:57.571336 638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:57.572290 638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:49:57.574290 731 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 12:49:57.580214 638 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 12:49:57.580327 638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260811 12:49:57.580439 638 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 12:49:57.583573 638 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 12:49:57.583652 638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.000s
I20260811 12:49:57.584103 731 tablet_bootstrap.cc:492] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec: Bootstrap starting.
I20260811 12:49:57.612809 731 log.cc:824] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec: Log is configured to *not* fsync() on all Append() calls
I20260811 12:49:57.622057 638 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:34015
I20260811 12:49:57.622092 923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:34015 every 8 connection(s)
I20260811 12:49:57.623328 638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:49:57.625506 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 638
I20260811 12:49:57.626093 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:45189
--local_ip_for_outbound_sockets=127.7.111.66
--tserver_master_addrs=127.7.111.126:34145
--webserver_port=37937
--webserver_interface=127.7.111.66
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:49:57.629223 731 tablet_bootstrap.cc:492] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:49:57.629782 731 tablet_bootstrap.cc:492] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec: Bootstrap complete.
I20260811 12:49:57.631172 731 ts_tablet_manager.cc:1397] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec: Time spent bootstrapping tablet: real 0.047s user 0.025s sys 0.020s
I20260811 12:49:57.633042 926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:57.630298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.65:53363)
0811 12:49:57.631353 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:57.631727 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":399,"threads_started":1}
W20260811 12:49:57.633934 924 heartbeater.cc:644] Failed to heartbeat to 127.7.111.126:34145 (0 consecutive failures): Network error: Failed to ping master at 127.7.111.126:34145: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
I20260811 12:49:57.635046 926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:57.634577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.65:36897)
0811 12:49:57.634777 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:57.634886 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 12:49:57.636155 926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:57.635717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.65:40947)
0811 12:49:57.635880 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:57.635985 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 12:49:57.636497 924 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 12:49:57.639617 731 raft_consensus.cc:348] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:49:57.640739 731 raft_consensus.cc:729] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d84abc316b84710a0a524e1e97af4ec, State: Initialized, Role: FOLLOWER
I20260811 12:49:57.641496 731 consensus_queue.cc:260] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:49:57.643126 926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:57.642797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.65:46885)
0811 12:49:57.642939 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:57.643009 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 12:49:57.643471 731 ts_tablet_manager.cc:1428] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260811 12:49:57.644328 926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:57.644025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.65:51679)
0811 12:49:57.644158 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:57.644220 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 12:49:57.964358 927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:57.964630 927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:57.964731 927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:49:57.971714 927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:57.971812 927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:57.971927 927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:49:57.981290 927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:45189
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=37937
--tserver_master_addrs=127.7.111.126:34145
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:57.982524 927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:57.983793 927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:57.996457 940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:57.996498 945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:57.996536 943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:57.997438 927 server_base.cc:1034] running on GCE node
I20260811 12:49:57.998024 927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:57.998945 927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:58.000147 927 hybrid_clock.cc:648] HybridClock initialized: now 1786452598000076 us; error 58 us; skew 500 ppm
I20260811 12:49:58.000438 927 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:57 dist-test-slave-f0tn krb5kdc[31130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452597, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:58.003531 927 init.cc:377] Logged in from keytab as kudu/127.7.111.66@KRBTEST.COM (short username kudu)
I20260811 12:49:58.005000 927 webserver.cc:460] Webserver started at http://127.7.111.66:37937/ using document root <none> and password file <none>
I20260811 12:49:58.005748 927 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:58.005867 927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:49:58.012737 927 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:49:58.015543 957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:58.016929 927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:49:58.017079 927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "2bc4568147c94a47862fb557a2abad4e"
format_stamp: "Formatted at 2026-08-11 12:49:52 on dist-test-slave-f0tn"
server_key: "a96dc100b483b944d76ada4098bfa456"
server_key_iv: "970c8d8b5c2ed518a4ba082c3466c681"
server_key_version: "encryptionkey@0"
I20260811 12:49:58.018075 927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:58.042374 927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:58.046320 927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:58.046598 927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:58.047580 927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:49:58.048897 927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:49:58.049000 927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:58.049081 927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:49:58.049134 927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:58.079162 927 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:45189
I20260811 12:49:58.079233 1171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:45189 every 8 connection(s)
I20260811 12:49:58.080681 927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:49:58.085578 1175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:58.083650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.66:57833)
0811 12:49:58.084394 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:58.084672 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":267,"threads_started":1}
W20260811 12:49:58.086232 1173 heartbeater.cc:644] Failed to heartbeat to 127.7.111.126:34145 (0 consecutive failures): Network error: Failed to ping master at 127.7.111.126:34145: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
I20260811 12:49:58.086989 1175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:58.086663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.66:45129)
0811 12:49:58.086817 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:58.086885 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 12:49:58.087463 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 927
I20260811 12:49:58.087805 1175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:58.087489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.66:45227)
0811 12:49:58.087626 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:58.087698 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 12:49:58.088019 1173 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 12:49:58.088078 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:43689
--local_ip_for_outbound_sockets=127.7.111.67
--tserver_master_addrs=127.7.111.126:34145
--webserver_port=44699
--webserver_interface=127.7.111.67
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:49:58.428078 1176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:49:58.428355 1176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:49:58.428491 1176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:49:58.435528 1176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:49:58.435634 1176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:49:58.435806 1176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:49:58.445399 1176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:43689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=44699
--tserver_master_addrs=127.7.111.126:34145
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:49:58.446607 1176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:49:58.447923 1176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:49:58.460597 1184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:58.460551 1186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:49:58.460635 1190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:49:58.461395 1176 server_base.cc:1034] running on GCE node
I20260811 12:49:58.461977 1176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:49:58.462845 1176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:49:58.464037 1176 hybrid_clock.cc:648] HybridClock initialized: now 1786452598463991 us; error 40 us; skew 500 ppm
I20260811 12:49:58.464329 1176 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:49:58 dist-test-slave-f0tn krb5kdc[31130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452598, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:49:58.467263 1176 init.cc:377] Logged in from keytab as kudu/127.7.111.67@KRBTEST.COM (short username kudu)
I20260811 12:49:58.468556 1176 webserver.cc:460] Webserver started at http://127.7.111.67:44699/ using document root <none> and password file <none>
I20260811 12:49:58.469192 1176 fs_manager.cc:359] Metadata directory not provided
I20260811 12:49:58.469313 1176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:49:58.474371 1176 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:49:58.476877 1198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:58.478214 1176 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:49:58.478341 1176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "1f7d34a9b95647459be576cbf304a48d"
format_stamp: "Formatted at 2026-08-11 12:49:52 on dist-test-slave-f0tn"
server_key: "ac49732928839896ecd9e206fa7ef32c"
server_key_iv: "0b3b2c170fc144658573c8512bf60836"
server_key_version: "encryptionkey@0"
I20260811 12:49:58.479403 1176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:49:58.499047 1176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:49:58.502923 1176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:49:58.503160 1176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:49:58.504136 1176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:49:58.505453 1176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:49:58.505542 1176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:58.505631 1176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:49:58.505685 1176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:49:58.537705 1176 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:43689
I20260811 12:49:58.537775 1400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:43689 every 8 connection(s)
I20260811 12:49:58.539096 1176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:49:58.543762 1404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:58.541967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.67:39093)
0811 12:49:58.542695 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:58.542925 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":261,"threads_started":1}
W20260811 12:49:58.544428 1401 heartbeater.cc:644] Failed to heartbeat to 127.7.111.126:34145 (0 consecutive failures): Network error: Failed to ping master at 127.7.111.126:34145: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
I20260811 12:49:58.545192 1404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:58.544888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.67:45343)
0811 12:49:58.545022 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:58.545090 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 12:49:58.545977 1404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:58.545703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.67:57051)
0811 12:49:58.545804 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:58.545876 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 12:49:58.546195 1401 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 12:49:58.548491 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 1176
I20260811 12:49:58.645923 1407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:58.645154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.65:59271)
0811 12:49:58.645627 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:58.645719 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":235,"threads_started":1}
I20260811 12:49:59.089267 1408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:59.088503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.66:56439)
0811 12:49:59.088957 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:59.089044 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":225,"threads_started":1}
I20260811 12:49:59.180255 1411 raft_consensus.cc:482] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:49:59.180519 1411 raft_consensus.cc:504] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:49:59.182204 1411 leader_election.cc:288] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2bc4568147c94a47862fb557a2abad4e (127.7.111.66:45189), 1f7d34a9b95647459be576cbf304a48d (127.7.111.67:43689)
Aug 11 12:49:59 dist-test-slave-f0tn krb5kdc[31130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452597, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for kudu/127.7.111.66@KRBTEST.COM
Aug 11 12:49:59 dist-test-slave-f0tn krb5kdc[31130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452597, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for kudu/127.7.111.67@KRBTEST.COM
I20260811 12:49:59.214388 1423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:59.182947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:45515 (local address 127.7.111.67:43689)
0811 12:49:59.195370 (+ 12423us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:59.195382 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:59.195620 (+ 238us) server_negotiation.cc:407] Connection header received
0811 12:49:59.197622 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:59.197630 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:59.197992 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:59.198273 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:59.200228 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:59.201540 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:59.203218 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:59.203813 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:59.203971 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:59.207868 (+ 3897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:59.207875 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:59.207886 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:59.207924 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:59.210794 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:59.211596 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:59.211604 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:59.211610 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:59.211761 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:59.212192 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:59.212196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:59.212198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:59.212689 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:59.212853 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:59.213639 (+ 786us) server_negotiation.cc:299] Negotiation successful
0811 12:49:59.213967 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12239,"thread_start_us":12133,"threads_started":1}
I20260811 12:49:59.214419 1413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:59.182519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:45189 (local address 127.7.111.65:48477)
0811 12:49:59.195235 (+ 12716us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:59.195291 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:59.195904 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:59.198429 (+ 2525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:59.198440 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:59.199321 (+ 881us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:49:59.199987 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:59.200010 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:59.201725 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:59.201729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:59.203073 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:59.203086 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:59.204062 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:59.204071 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:59.204313 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:59.205064 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:49:59.205081 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:49:59.207477 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:49:59.210813 (+ 3336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:59.210822 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:59.210836 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:59.211450 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:59.211922 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:59.211929 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:59.211931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:59.212093 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:59.212832 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:49:59.212844 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:49:59.213149 (+ 305us) client_negotiation.cc:769] Sending connection context
0811 12:49:59.213409 (+ 260us) client_negotiation.cc:240] Negotiation successful
0811 12:49:59.213975 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12554,"mutex_wait_us":184,"spinlock_wait_cycles":50816,"thread_start_us":12782,"threads_started":1}
I20260811 12:49:59.214414 1421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:59.182809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:48477 (local address 127.7.111.66:45189)
0811 12:49:59.194925 (+ 12116us) server_negotiation.cc:206] Beginning negotiation
0811 12:49:59.194945 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 12:49:59.195472 (+ 527us) server_negotiation.cc:407] Connection header received
0811 12:49:59.197662 (+ 2190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:49:59.197671 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:49:59.197996 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:49:59.198274 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:49:59.200201 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:59.201543 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:59.203207 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:49:59.203816 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:49:59.203983 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:59.207781 (+ 3798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:49:59.207793 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:49:59.207807 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:49:59.207843 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:49:59.210662 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:59.211608 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:59.211616 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:59.211624 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:59.211791 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:49:59.212212 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:49:59.212217 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:49:59.212221 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:49:59.212695 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:49:59.212865 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 12:49:59.213660 (+ 795us) server_negotiation.cc:299] Negotiation successful
0811 12:49:59.214037 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11927,"thread_start_us":11749,"threads_started":1}
I20260811 12:49:59.214829 1415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:59.182668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:43689 (local address 127.7.111.65:45515)
0811 12:49:59.195488 (+ 12820us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:59.195515 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 12:49:59.195890 (+ 375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:49:59.198429 (+ 2539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:49:59.198440 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:49:59.199320 (+ 880us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:49:59.199985 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:59.200007 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:59.201727 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:59.201731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:59.203078 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:49:59.203091 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:49:59.204020 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:49:59.204027 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:49:59.204309 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:49:59.205012 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:49:59.205041 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:49:59.207642 (+ 2601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:49:59.210931 (+ 3289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:59.210937 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:59.210941 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:59.211475 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:59.211878 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:49:59.211883 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:49:59.211887 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:49:59.212089 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:49:59.212832 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:49:59.212840 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:49:59.213150 (+ 310us) client_negotiation.cc:769] Sending connection context
0811 12:49:59.213409 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 12:49:59.214122 (+ 713us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12677,"mutex_wait_us":296,"spinlock_wait_cycles":22656,"thread_start_us":12822,"threads_started":1}
W20260811 12:49:59.217614 723 leader_election.cc:341] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2bc4568147c94a47862fb557a2abad4e (127.7.111.66:45189): Not found: Tablet not found: 709ee1329e964c988977413a0baefdda
W20260811 12:49:59.217888 725 leader_election.cc:341] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1f7d34a9b95647459be576cbf304a48d (127.7.111.67:43689): Not found: Tablet not found: 709ee1329e964c988977413a0baefdda
I20260811 12:49:59.218016 725 leader_election.cc:302] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4d84abc316b84710a0a524e1e97af4ec; no voters: 1f7d34a9b95647459be576cbf304a48d, 2bc4568147c94a47862fb557a2abad4e
I20260811 12:49:59.218410 1411 raft_consensus.cc:2726] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:49:59.547514 1424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:59.546675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.67:35477)
0811 12:49:59.547144 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:59.547294 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":220,"threads_started":1}
I20260811 12:49:59.647208 1415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:49:59.646803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.65:37069)
0811 12:49:59.646966 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 12:49:59.647062 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 12:50:00.090782 1425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:00.090095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.66:47953)
0811 12:50:00.090540 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:00.090629 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":227,"threads_started":1}
I20260811 12:50:00.548954 1428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:00.548279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.67:55569)
0811 12:50:00.548720 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:00.548803 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":230,"threads_started":1}
I20260811 12:50:00.648797 1430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:00.648030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.65:58919)
0811 12:50:00.648524 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:00.648610 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":222,"threads_started":1}
I20260811 12:50:01.092418 1431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:01.091655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.66:49515)
0811 12:50:01.092132 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:01.092238 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":248,"threads_started":1}
I20260811 12:50:01.393079 1433 raft_consensus.cc:482] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:50:01.393262 1433 raft_consensus.cc:504] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:50:01.393968 1433 leader_election.cc:288] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2bc4568147c94a47862fb557a2abad4e (127.7.111.66:45189), 1f7d34a9b95647459be576cbf304a48d (127.7.111.67:43689)
W20260811 12:50:01.394928 723 leader_election.cc:341] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2bc4568147c94a47862fb557a2abad4e (127.7.111.66:45189): Not found: Tablet not found: 709ee1329e964c988977413a0baefdda
W20260811 12:50:01.395118 725 leader_election.cc:341] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1f7d34a9b95647459be576cbf304a48d (127.7.111.67:43689): Not found: Tablet not found: 709ee1329e964c988977413a0baefdda
I20260811 12:50:01.395227 725 leader_election.cc:302] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4d84abc316b84710a0a524e1e97af4ec; no voters: 1f7d34a9b95647459be576cbf304a48d, 2bc4568147c94a47862fb557a2abad4e
I20260811 12:50:01.395448 1433 raft_consensus.cc:2726] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:50:01.550642 1437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:01.549865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.67:56615)
0811 12:50:01.550376 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:01.550468 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":261,"threads_started":1}
I20260811 12:50:01.650465 1438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:01.649675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.65:49683)
0811 12:50:01.650153 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:01.650271 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":233,"threads_started":1}
I20260811 12:50:02.094182 1439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:02.093271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.66:51443)
0811 12:50:02.093810 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:02.093927 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":253,"threads_started":1}
I20260811 12:50:02.552237 1443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:02.551522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.67:59673)
0811 12:50:02.551955 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:02.552039 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":209,"threads_started":1}
I20260811 12:50:02.652426 1444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:02.651678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.65:41197)
0811 12:50:02.652148 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:02.652245 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":239,"threads_started":1}
I20260811 12:50:03.095924 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:03.095182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.66:43945)
0811 12:50:03.095651 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:03.095739 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":240,"threads_started":1}
I20260811 12:50:03.553930 1446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:03.553184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.67:37913)
0811 12:50:03.553668 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:03.553757 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":273,"threads_started":1}
I20260811 12:50:03.654042 1447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:03.653307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.65:34145)
0811 12:50:03.653790 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:03.653873 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":255,"threads_started":1}
I20260811 12:50:04.097602 1451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:04.096837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.66:38551)
0811 12:50:04.097295 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:04.097405 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":230,"threads_started":1}
I20260811 12:50:04.555547 1454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:04.554819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.67:54867)
0811 12:50:04.555286 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:04.555369 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":238,"threads_started":1}
I20260811 12:50:04.655658 1456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:04.654908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.65:49763)
0811 12:50:04.655360 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:04.655454 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":231,"threads_started":1}
I20260811 12:50:04.710670 1457 raft_consensus.cc:482] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:50:04.710906 1457 raft_consensus.cc:504] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:50:04.711647 1457 leader_election.cc:288] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2bc4568147c94a47862fb557a2abad4e (127.7.111.66:45189), 1f7d34a9b95647459be576cbf304a48d (127.7.111.67:43689)
W20260811 12:50:04.712590 725 leader_election.cc:341] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1f7d34a9b95647459be576cbf304a48d (127.7.111.67:43689): Not found: Tablet not found: 709ee1329e964c988977413a0baefdda
W20260811 12:50:04.712764 723 leader_election.cc:341] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2bc4568147c94a47862fb557a2abad4e (127.7.111.66:45189): Not found: Tablet not found: 709ee1329e964c988977413a0baefdda
I20260811 12:50:04.712898 723 leader_election.cc:302] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4d84abc316b84710a0a524e1e97af4ec; no voters: 1f7d34a9b95647459be576cbf304a48d, 2bc4568147c94a47862fb557a2abad4e
I20260811 12:50:04.713136 1457 raft_consensus.cc:2726] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:50:05.099361 1458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:05.098502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.66:34769)
0811 12:50:05.099070 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:05.099171 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":259,"threads_started":1}
I20260811 12:50:05.557166 1459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:05.556372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.67:42317)
0811 12:50:05.556897 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:05.556982 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":225,"threads_started":1}
I20260811 12:50:05.657135 1460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:05.656461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.65:48601)
0811 12:50:05.656890 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:05.656971 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":221,"threads_started":1}
I20260811 12:50:06.100920 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:06.100252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.66:42851)
0811 12:50:06.100644 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:06.100747 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":202,"threads_started":1}
I20260811 12:50:06.558791 1464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:06.558083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.67:45345)
0811 12:50:06.558516 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:06.558597 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":210,"threads_started":1}
I20260811 12:50:06.659019 1465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:06.658140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.65:56617)
0811 12:50:06.658745 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:06.658833 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":363,"threads_started":1}
I20260811 12:50:07.102536 1466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:07.101827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.66:34597)
0811 12:50:07.102288 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:07.102372 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":212,"threads_started":1}
I20260811 12:50:07.560595 1467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:07.559756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.67:35805)
0811 12:50:07.560339 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:07.560427 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":315,"threads_started":1}
I20260811 12:50:07.610121 1472 raft_consensus.cc:482] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:50:07.610328 1472 raft_consensus.cc:504] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:50:07.610967 1472 leader_election.cc:288] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2bc4568147c94a47862fb557a2abad4e (127.7.111.66:45189), 1f7d34a9b95647459be576cbf304a48d (127.7.111.67:43689)
W20260811 12:50:07.611903 725 leader_election.cc:341] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1f7d34a9b95647459be576cbf304a48d (127.7.111.67:43689): Not found: Tablet not found: 709ee1329e964c988977413a0baefdda
W20260811 12:50:07.612087 723 leader_election.cc:341] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2bc4568147c94a47862fb557a2abad4e (127.7.111.66:45189): Not found: Tablet not found: 709ee1329e964c988977413a0baefdda
I20260811 12:50:07.612216 723 leader_election.cc:302] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4d84abc316b84710a0a524e1e97af4ec; no voters: 1f7d34a9b95647459be576cbf304a48d, 2bc4568147c94a47862fb557a2abad4e
I20260811 12:50:07.612462 1472 raft_consensus.cc:2726] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:50:07.661024 1473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:07.660092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.65:40543)
0811 12:50:07.660660 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:07.660816 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":307,"threads_started":1}
I20260811 12:50:08.104161 1475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:08.103400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.66:36677)
0811 12:50:08.103892 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:08.103976 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":226,"threads_started":1}
I20260811 12:50:08.562273 1476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:08.561485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.67:49513)
0811 12:50:08.561940 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:08.562076 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":219,"threads_started":1}
I20260811 12:50:08.662952 1477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:08.662158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.65:57101)
0811 12:50:08.662664 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:08.662764 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":251,"threads_started":1}
I20260811 12:50:09.105753 1478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:09.104994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.66:60973)
0811 12:50:09.105501 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:09.105581 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":248,"threads_started":1}
I20260811 12:50:09.563829 1479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:09.563111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.67:36115)
0811 12:50:09.563562 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:09.563649 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":222,"threads_started":1}
I20260811 12:50:09.664670 1480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:09.663836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.65:57209)
0811 12:50:09.664370 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:09.664467 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":287,"threads_started":1}
I20260811 12:50:10.107326 1481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:10.106611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.66:42923)
0811 12:50:10.107061 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:10.107143 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":234,"threads_started":1}
I20260811 12:50:10.294632 1483 raft_consensus.cc:482] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:50:10.294817 1483 raft_consensus.cc:504] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc4568147c94a47862fb557a2abad4e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 45189 } } peers { permanent_uuid: "4d84abc316b84710a0a524e1e97af4ec" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 34015 } } peers { permanent_uuid: "1f7d34a9b95647459be576cbf304a48d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 43689 } }
I20260811 12:50:10.295459 1483 leader_election.cc:288] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2bc4568147c94a47862fb557a2abad4e (127.7.111.66:45189), 1f7d34a9b95647459be576cbf304a48d (127.7.111.67:43689)
W20260811 12:50:10.296366 723 leader_election.cc:341] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2bc4568147c94a47862fb557a2abad4e (127.7.111.66:45189): Not found: Tablet not found: 709ee1329e964c988977413a0baefdda
W20260811 12:50:10.296557 725 leader_election.cc:341] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1f7d34a9b95647459be576cbf304a48d (127.7.111.67:43689): Not found: Tablet not found: 709ee1329e964c988977413a0baefdda
I20260811 12:50:10.296684 725 leader_election.cc:302] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4d84abc316b84710a0a524e1e97af4ec; no voters: 1f7d34a9b95647459be576cbf304a48d, 2bc4568147c94a47862fb557a2abad4e
I20260811 12:50:10.296927 1483 raft_consensus.cc:2726] T 709ee1329e964c988977413a0baefdda P 4d84abc316b84710a0a524e1e97af4ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:50:10.565531 1484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:10.564736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.67:42477)
0811 12:50:10.565220 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:10.565309 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":247,"threads_started":1}
I20260811 12:50:10.666566 1485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:10.665714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.65:44903)
0811 12:50:10.666263 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:10.666366 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":275,"threads_started":1}
I20260811 12:50:11.108965 1489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:11.108219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.66:51739)
0811 12:50:11.108693 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:11.108802 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":240,"threads_started":1}
I20260811 12:50:11.567123 1490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:11.566363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.67:60635)
0811 12:50:11.566870 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:11.566961 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":260,"threads_started":1}
I20260811 12:50:11.668171 1493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:11.667437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.65:60587)
0811 12:50:11.667902 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:11.667993 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":237,"threads_started":1}
I20260811 12:50:12.110530 1495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:12.109787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.66:38011)
0811 12:50:12.110252 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:12.110336 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":233,"threads_started":1}
I20260811 12:50:12.568709 1497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:12.567955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.67:45059)
0811 12:50:12.568402 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:12.568494 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":223,"threads_started":1}
I20260811 12:50:12.670360 1498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:12.669506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.65:54407)
0811 12:50:12.669985 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:12.670102 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":255,"threads_started":1}
I20260811 12:50:13.112299 1500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:13.111537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.66:43547)
0811 12:50:13.112019 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:13.112116 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":228,"threads_started":1}
I20260811 12:50:13.559985 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 638
I20260811 12:50:13.570569 1504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:13.569794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:34145 (local address 127.7.111.67:43847)
0811 12:50:13.570281 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:13.570406 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:34145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":259,"threads_started":1}
I20260811 12:50:13.579481 1503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:13.561332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38454 (local address 127.7.111.65:34015)
0811 12:50:13.561956 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:13.561967 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:13.562287 (+ 320us) server_negotiation.cc:407] Connection header received
0811 12:50:13.562516 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:13.562529 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:13.562827 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:13.562984 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:50:13.564454 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:13.565518 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:13.566824 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:13.567406 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:13.567525 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:13.573974 (+ 6449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:50:13.573987 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:50:13.574003 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:50:13.574044 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:50:13.576659 (+ 2615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:13.577319 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:13.577327 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:13.577329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:13.577528 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:13.577899 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:13.577904 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:13.577908 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:13.578286 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:50:13.578597 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:13.578907 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 12:50:13.579173 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":260,"threads_started":1}
I20260811 12:50:13.580731 917 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:50:13.697786 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 638
I20260811 12:50:13.726356 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 927
I20260811 12:50:13.739508 1508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:13.727058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49084 (local address 127.7.111.66:45189)
0811 12:50:13.727539 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:13.727544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:13.727562 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:50:13.727618 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:13.727622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:13.727686 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:13.727799 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:50:13.729063 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:13.729895 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:13.731075 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:13.731500 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:13.731591 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:13.734372 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:50:13.734377 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:50:13.734380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:50:13.734417 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:50:13.736814 (+ 2397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:13.737757 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:13.737762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:13.737764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:13.737897 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:13.738199 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:13.738202 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:13.738204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:13.738475 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:50:13.738668 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:13.738907 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 12:50:13.739082 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":258,"threads_started":1}
I20260811 12:50:13.740252 1161 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:50:13.823915 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 927
I20260811 12:50:13.851548 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 1176
I20260811 12:50:13.864554 1510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:13.852335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42192 (local address 127.7.111.67:43689)
0811 12:50:13.852877 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:13.852881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:13.852899 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:50:13.852957 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:13.852960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:13.853030 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:13.853123 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:50:13.854572 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:13.855347 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:13.856506 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:13.856923 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:13.857011 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:13.859751 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:50:13.859756 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:50:13.859759 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:50:13.859791 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:50:13.862346 (+ 2555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:13.863015 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:13.863018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:13.863021 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:13.863157 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:13.863430 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:13.863432 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:13.863434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:13.863680 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:50:13.863879 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:13.864091 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 12:50:13.864268 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":267,"threads_started":1}
I20260811 12:50:13.865262 1393 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:50:13.948199 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 1176
2026-08-11T12:50:13Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24810 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:50:14 dist-test-slave-f0tn krb5kdc[1521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:50:14 dist-test-slave-f0tn krb5kdc[1521](info): set up 2 sockets
Aug 11 12:50:14 dist-test-slave-f0tn krb5kdc[1521](info): commencing operation
krb5kdc: starting...
W20260811 12:50:15.215938 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:50:15 dist-test-slave-f0tn krb5kdc[1521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:50:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:50:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:50:15.520378 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:40455
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:44687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:40455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:50:15.862176 1545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:50:15.862460 1545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:50:15.862599 1545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:50:15.869725 1545 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 12:50:15.869824 1545 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 12:50:15.869887 1545 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 12:50:15.869956 1545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:50:15.870035 1545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:50:15.870121 1545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:50:15.870164 1545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:50:15.870237 1545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:50:15.880024 1545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:44687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:40455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:40455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:50:15.881429 1545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:50:15.882845 1545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:50:15.894773 1551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:50:15.894804 1554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:50:15.894742 1552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:50:15.895586 1545 server_base.cc:1034] running on GCE node
I20260811 12:50:15.896404 1545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:50:15.897912 1545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:50:15.899158 1545 hybrid_clock.cc:648] HybridClock initialized: now 1786452615899059 us; error 40 us; skew 500 ppm
I20260811 12:50:15.899494 1545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:50:15 dist-test-slave-f0tn krb5kdc[1521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452615, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:50:15.902665 1545 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:50:15.904250 1545 webserver.cc:460] Webserver started at http://127.7.111.126:45933/ using document root <none> and password file <none>
I20260811 12:50:15.904970 1545 fs_manager.cc:359] Metadata directory not provided
I20260811 12:50:15.905076 1545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:50:15.905419 1545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:50:15.908522 1545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "17ccc579f9934760a2dcd371e1c026fb"
format_stamp: "Formatted at 2026-08-11 12:50:15 on dist-test-slave-f0tn"
server_key: "7183e2e070a64cf53f27eb77929542e5"
server_key_iv: "4aad647f02b61c7b03459e2a3b25302e"
server_key_version: "encryptionkey@0"
I20260811 12:50:15.909332 1545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "17ccc579f9934760a2dcd371e1c026fb"
format_stamp: "Formatted at 2026-08-11 12:50:15 on dist-test-slave-f0tn"
server_key: "7183e2e070a64cf53f27eb77929542e5"
server_key_iv: "4aad647f02b61c7b03459e2a3b25302e"
server_key_version: "encryptionkey@0"
I20260811 12:50:15.914707 1545 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 12:50:15.918242 1563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:50:15.919605 1545 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:50:15.919791 1545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "17ccc579f9934760a2dcd371e1c026fb"
format_stamp: "Formatted at 2026-08-11 12:50:15 on dist-test-slave-f0tn"
server_key: "7183e2e070a64cf53f27eb77929542e5"
server_key_iv: "4aad647f02b61c7b03459e2a3b25302e"
server_key_version: "encryptionkey@0"
I20260811 12:50:15.920007 1545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:50:15.932794 1545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:50:15.936795 1545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:50:15.937041 1545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:50:15.954727 1545 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:40455
I20260811 12:50:15.954703 1658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:40455 every 8 connection(s)
I20260811 12:50:15.956058 1545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:50:15.959862 1659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:50:15.962594 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 1545
I20260811 12:50:15.962821 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:50:15.963634 7613 external_mini_cluster.cc:1428] Setting key 5ba9c8ca5a8c66df150dc15db8bf68cf
I20260811 12:50:15.971704 1659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb: Bootstrap starting.
I20260811 12:50:15.974774 1659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb: Neither blocks nor log segments found. Creating new log.
I20260811 12:50:15.975783 1659 log.cc:824] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb: Log is configured to *not* fsync() on all Append() calls
Aug 11 12:50:15 dist-test-slave-f0tn krb5kdc[1521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:50:15.979197 1659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb: No bootstrap required, opened a new log
I20260811 12:50:15.986483 1662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:15.965846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33648 (local address 127.7.111.126:40455)
0811 12:50:15.966593 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:15.966606 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:15.966652 (+ 46us) server_negotiation.cc:407] Connection header received
0811 12:50:15.968306 (+ 1654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:15.968333 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:15.968709 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:15.969074 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:50:15.970580 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:15.971909 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:15.973200 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:15.973891 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:15.974055 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:15.978785 (+ 4730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:50:15.978793 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:50:15.978808 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:50:15.978844 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:50:15.982278 (+ 3434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:15.983023 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:15.983030 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:15.983037 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:15.983215 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:15.983618 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:15.983622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:15.983624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:15.984178 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:50:15.984476 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:15.984950 (+ 474us) server_negotiation.cc:299] Negotiation successful
0811 12:50:15.985424 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":322,"threads_started":1}
I20260811 12:50:15.987293 1659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ccc579f9934760a2dcd371e1c026fb" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40455 } }
I20260811 12:50:15.987665 1659 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:50:15.987790 1659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17ccc579f9934760a2dcd371e1c026fb, State: Initialized, Role: FOLLOWER
I20260811 12:50:15.988443 1659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ccc579f9934760a2dcd371e1c026fb" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40455 } }
I20260811 12:50:15.988762 1659 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:50:15.988901 1659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:50:15.989084 1659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:50:15.991387 1659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ccc579f9934760a2dcd371e1c026fb" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40455 } }
I20260811 12:50:15.991852 1659 leader_election.cc:302] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17ccc579f9934760a2dcd371e1c026fb; no voters:
I20260811 12:50:15.992417 1659 leader_election.cc:288] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:50:15.992543 1668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:50:15.993624 1668 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 1 LEADER]: Becoming Leader. State: Replica: 17ccc579f9934760a2dcd371e1c026fb, State: Running, Role: LEADER
I20260811 12:50:15.994187 1668 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ccc579f9934760a2dcd371e1c026fb" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40455 } }
I20260811 12:50:15.994606 1659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:50:15.996817 1670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17ccc579f9934760a2dcd371e1c026fb. Latest consensus state: current_term: 1 leader_uuid: "17ccc579f9934760a2dcd371e1c026fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ccc579f9934760a2dcd371e1c026fb" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40455 } } }
I20260811 12:50:15.997219 1670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [sys.catalog]: This master's current role is: LEADER
I20260811 12:50:15.997879 1669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17ccc579f9934760a2dcd371e1c026fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ccc579f9934760a2dcd371e1c026fb" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40455 } } }
I20260811 12:50:15.998150 1669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [sys.catalog]: This master's current role is: LEADER
I20260811 12:50:15.998227 1679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:50:16.003763 1679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:50:16.012841 1679 catalog_manager.cc:1347] Generated new cluster ID: f96b0f04889143ec87815c16db24b363
I20260811 12:50:16.012948 1679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:50:16.023818 1679 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:50:16.025190 1679 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:50:16.036514 1679 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb: Generated new TSK 0
I20260811 12:50:16.037351 1679 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.111.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
I20260811 12:50:16.120875 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40455
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:44687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:50:16.538793 1696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:50:16.539052 1696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:50:16.539142 1696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:50:16.546321 1696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:50:16.546437 1696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:50:16.546566 1696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:50:16.556452 1696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:44687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:50:16.557854 1696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:50:16.559234 1696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:50:16.572688 1703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:50:16.572597 1711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:50:16.572652 1708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:50:16.573618 1696 server_base.cc:1034] running on GCE node
I20260811 12:50:16.574221 1696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:50:16.575177 1696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:50:16.576371 1696 hybrid_clock.cc:648] HybridClock initialized: now 1786452616576312 us; error 57 us; skew 500 ppm
I20260811 12:50:16.576691 1696 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:50:16 dist-test-slave-f0tn krb5kdc[1521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452616, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:50:16.579813 1696 init.cc:377] Logged in from keytab as kudu/127.7.111.65@KRBTEST.COM (short username kudu)
I20260811 12:50:16.581337 1696 webserver.cc:460] Webserver started at http://127.7.111.65:34863/ using document root <none> and password file <none>
I20260811 12:50:16.582082 1696 fs_manager.cc:359] Metadata directory not provided
I20260811 12:50:16.582197 1696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:50:16.582526 1696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:50:16.585590 1696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "c2766f606ebd4d81a30334721da429eb"
format_stamp: "Formatted at 2026-08-11 12:50:16 on dist-test-slave-f0tn"
server_key: "d98424a089994169da8f3065415057f6"
server_key_iv: "9a4105b73c3f2ef0ec114fab74f642ea"
server_key_version: "encryptionkey@0"
I20260811 12:50:16.586330 1696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "c2766f606ebd4d81a30334721da429eb"
format_stamp: "Formatted at 2026-08-11 12:50:16 on dist-test-slave-f0tn"
server_key: "d98424a089994169da8f3065415057f6"
server_key_iv: "9a4105b73c3f2ef0ec114fab74f642ea"
server_key_version: "encryptionkey@0"
I20260811 12:50:16.591764 1696 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:50:16.595710 1721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:50:16.597445 1696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:50:16.597620 1696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "c2766f606ebd4d81a30334721da429eb"
format_stamp: "Formatted at 2026-08-11 12:50:16 on dist-test-slave-f0tn"
server_key: "d98424a089994169da8f3065415057f6"
server_key_iv: "9a4105b73c3f2ef0ec114fab74f642ea"
server_key_version: "encryptionkey@0"
I20260811 12:50:16.597819 1696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:50:16.629525 1696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:50:16.633816 1696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:50:16.634080 1696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:50:16.635072 1696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:50:16.636433 1696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:50:16.636507 1696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:50:16.636626 1696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:50:16.636675 1696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:50:16.666851 1696 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:44679
I20260811 12:50:16.666910 1906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:44679 every 8 connection(s)
I20260811 12:50:16.668246 1696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:50:16.678512 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 1696
I20260811 12:50:16.678785 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:50:16.679632 7613 external_mini_cluster.cc:1428] Setting key f3ae0e8aa3b36b43f0a51a4f6b7a7ddc
Aug 11 12:50:16 dist-test-slave-f0tn krb5kdc[1521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452616, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:50:16.688212 1915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:16.670778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:44235 (local address 127.7.111.126:40455)
0811 12:50:16.671245 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:16.671249 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:16.672198 (+ 949us) server_negotiation.cc:407] Connection header received
0811 12:50:16.674163 (+ 1965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:16.674167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:16.674241 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:16.674325 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:50:16.675810 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:16.676537 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:16.678076 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:16.678460 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:16.678552 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:16.682333 (+ 3781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:50:16.682339 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:50:16.682342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:50:16.682379 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:50:16.685344 (+ 2965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:16.686043 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:16.686048 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:16.686051 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:16.686217 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:16.686581 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:16.686586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:16.686589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:16.686911 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:50:16.687047 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:16.687649 (+ 602us) server_negotiation.cc:299] Negotiation successful
0811 12:50:16.687874 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":226,"threads_started":1}
I20260811 12:50:16.689035 1916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:16.671354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.65:44235)
0811 12:50:16.672038 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:16.672075 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:50:16.673893 (+ 1818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:50:16.674516 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:50:16.674524 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:50:16.675014 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:50:16.675662 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:16.675680 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:16.676684 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:16.676688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:16.677948 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:16.677960 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:16.678706 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:16.678714 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:16.679017 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:16.679708 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:50:16.679738 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:50:16.682089 (+ 2351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:50:16.685485 (+ 3396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:50:16.685491 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:50:16.685503 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:50:16.685922 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:50:16.686307 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:50:16.686310 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:50:16.686313 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:50:16.686481 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:50:16.687014 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:50:16.687019 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:50:16.687306 (+ 287us) client_negotiation.cc:769] Sending connection context
0811 12:50:16.687519 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 12:50:16.687894 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":275,"threads_started":1}
I20260811 12:50:16.690452 1909 heartbeater.cc:347] Connected to a master server at 127.7.111.126:40455
I20260811 12:50:16.690748 1909 heartbeater.cc:464] Registering TS with master...
I20260811 12:50:16.691438 1909 heartbeater.cc:511] Master 127.7.111.126:40455 requested a full tablet report, sending...
I20260811 12:50:16.693483 1589 ts_manager.cc:194] Registered new tserver with Master: c2766f606ebd4d81a30334721da429eb (127.7.111.65:44679)
I20260811 12:50:16.694990 1589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:44235
WARNING: no policy specified for kudu/127.7.111.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
I20260811 12:50:16.746212 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40455
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:44687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:50:17.084864 1919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:50:17.085160 1919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:50:17.085284 1919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:50:17.092376 1919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:50:17.092489 1919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:50:17.092641 1919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:50:17.102893 1919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:44687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:50:17.104229 1919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:50:17.105682 1919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:50:17.118506 1930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:50:17.118563 1927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:50:17.118736 1925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:50:17.119697 1919 server_base.cc:1034] running on GCE node
I20260811 12:50:17.120322 1919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:50:17.121322 1919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:50:17.122537 1919 hybrid_clock.cc:648] HybridClock initialized: now 1786452617122482 us; error 50 us; skew 500 ppm
I20260811 12:50:17.122852 1919 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:50:17 dist-test-slave-f0tn krb5kdc[1521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452617, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:50:17.126084 1919 init.cc:377] Logged in from keytab as kudu/127.7.111.66@KRBTEST.COM (short username kudu)
I20260811 12:50:17.127483 1919 webserver.cc:460] Webserver started at http://127.7.111.66:34271/ using document root <none> and password file <none>
I20260811 12:50:17.128149 1919 fs_manager.cc:359] Metadata directory not provided
I20260811 12:50:17.128252 1919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:50:17.128509 1919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:50:17.131503 1919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "66dcdc14b4cc4517b3b044a53d900d0d"
format_stamp: "Formatted at 2026-08-11 12:50:17 on dist-test-slave-f0tn"
server_key: "033a81ef562d2557a07e95f58152f6f5"
server_key_iv: "cccf2fa4f344d0e2f7f390517e0c37f0"
server_key_version: "encryptionkey@0"
I20260811 12:50:17.132222 1919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "66dcdc14b4cc4517b3b044a53d900d0d"
format_stamp: "Formatted at 2026-08-11 12:50:17 on dist-test-slave-f0tn"
server_key: "033a81ef562d2557a07e95f58152f6f5"
server_key_iv: "cccf2fa4f344d0e2f7f390517e0c37f0"
server_key_version: "encryptionkey@0"
I20260811 12:50:17.137295 1919 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:50:17.140802 1937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:50:17.142210 1919 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:50:17.142364 1919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "66dcdc14b4cc4517b3b044a53d900d0d"
format_stamp: "Formatted at 2026-08-11 12:50:17 on dist-test-slave-f0tn"
server_key: "033a81ef562d2557a07e95f58152f6f5"
server_key_iv: "cccf2fa4f344d0e2f7f390517e0c37f0"
server_key_version: "encryptionkey@0"
I20260811 12:50:17.142545 1919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:50:17.157146 1919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:50:17.161109 1919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:50:17.161415 1919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:50:17.162400 1919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:50:17.163758 1919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:50:17.163847 1919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:50:17.163964 1919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:50:17.164017 1919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:50:17.194928 1919 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:42023
I20260811 12:50:17.194986 2111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:42023 every 8 connection(s)
I20260811 12:50:17.196271 1919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:50:17.206614 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 1919
I20260811 12:50:17.206805 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:50:17.207274 7613 external_mini_cluster.cc:1428] Setting key 2910abc57c070f7d8a54bfdfab78dcdf
Aug 11 12:50:17 dist-test-slave-f0tn krb5kdc[1521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452617, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:50:17.217016 2116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:17.198884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:38949 (local address 127.7.111.126:40455)
0811 12:50:17.199370 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:17.199374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:17.200322 (+ 948us) server_negotiation.cc:407] Connection header received
0811 12:50:17.202321 (+ 1999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:17.202328 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:17.202409 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:17.202505 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:50:17.204078 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:17.204821 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:17.206469 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:17.206962 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:17.207081 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:17.210796 (+ 3715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:50:17.210803 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:50:17.210808 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:50:17.210849 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:50:17.213875 (+ 3026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:17.214670 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:17.214676 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:17.214679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:17.214855 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:17.215279 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:17.215283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:17.215286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:17.215596 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:50:17.215737 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:17.216422 (+ 685us) server_negotiation.cc:299] Negotiation successful
0811 12:50:17.216641 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":245,"threads_started":1}
I20260811 12:50:17.217883 2118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:17.199346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.66:38949)
0811 12:50:17.200154 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:17.200196 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:50:17.202043 (+ 1847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:50:17.202797 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:50:17.202806 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:50:17.203325 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:50:17.203907 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:17.203924 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:17.204978 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:17.204982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:17.206327 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:17.206340 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:17.207195 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:17.207203 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:17.207454 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:17.208182 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:50:17.208213 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:50:17.210552 (+ 2339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:50:17.214017 (+ 3465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:50:17.214025 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:50:17.214039 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:50:17.214549 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:50:17.214951 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:50:17.214954 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:50:17.214956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:50:17.215157 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:50:17.215704 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:50:17.215709 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:50:17.216045 (+ 336us) client_negotiation.cc:769] Sending connection context
0811 12:50:17.216287 (+ 242us) client_negotiation.cc:240] Negotiation successful
0811 12:50:17.216697 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":300,"threads_started":1}
I20260811 12:50:17.219331 2113 heartbeater.cc:347] Connected to a master server at 127.7.111.126:40455
I20260811 12:50:17.219664 2113 heartbeater.cc:464] Registering TS with master...
I20260811 12:50:17.220557 2113 heartbeater.cc:511] Master 127.7.111.126:40455 requested a full tablet report, sending...
I20260811 12:50:17.222193 1589 ts_manager.cc:194] Registered new tserver with Master: 66dcdc14b4cc4517b3b044a53d900d0d (127.7.111.66:42023)
I20260811 12:50:17.223008 1589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:38949
WARNING: no policy specified for kudu/127.7.111.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
I20260811 12:50:17.275841 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40455
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:44687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:50:17.622123 2126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:50:17.622416 2126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:50:17.622565 2126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:50:17.629837 2126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:50:17.629951 2126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:50:17.630107 2126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:50:17.639674 2126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:44687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:50:17.640928 2126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:50:17.642326 2126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:50:17.655424 2137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:50:17.655498 2136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:50:17.655534 2140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:50:17.656489 2126 server_base.cc:1034] running on GCE node
I20260811 12:50:17.657143 2126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:50:17.658195 2126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:50:17.659410 2126 hybrid_clock.cc:648] HybridClock initialized: now 1786452617659361 us; error 36 us; skew 500 ppm
I20260811 12:50:17.659775 2126 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:50:17 dist-test-slave-f0tn krb5kdc[1521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452617, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:50:17.663096 2126 init.cc:377] Logged in from keytab as kudu/127.7.111.67@KRBTEST.COM (short username kudu)
I20260811 12:50:17.664664 2126 webserver.cc:460] Webserver started at http://127.7.111.67:33979/ using document root <none> and password file <none>
I20260811 12:50:17.665437 2126 fs_manager.cc:359] Metadata directory not provided
I20260811 12:50:17.665587 2126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:50:17.665926 2126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:50:17.669071 2126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "d99382592a4347488d921cece4145022"
format_stamp: "Formatted at 2026-08-11 12:50:17 on dist-test-slave-f0tn"
server_key: "c4d5aacbceee4de22be82aaed2c87f01"
server_key_iv: "a3e8a8e6e6ecc2a199b9724cd9d36c45"
server_key_version: "encryptionkey@0"
I20260811 12:50:17.669904 2126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "d99382592a4347488d921cece4145022"
format_stamp: "Formatted at 2026-08-11 12:50:17 on dist-test-slave-f0tn"
server_key: "c4d5aacbceee4de22be82aaed2c87f01"
server_key_iv: "a3e8a8e6e6ecc2a199b9724cd9d36c45"
server_key_version: "encryptionkey@0"
I20260811 12:50:17.675243 2126 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260811 12:50:17.678887 2148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:50:17.680160 2126 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:50:17.680330 2126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "d99382592a4347488d921cece4145022"
format_stamp: "Formatted at 2026-08-11 12:50:17 on dist-test-slave-f0tn"
server_key: "c4d5aacbceee4de22be82aaed2c87f01"
server_key_iv: "a3e8a8e6e6ecc2a199b9724cd9d36c45"
server_key_version: "encryptionkey@0"
I20260811 12:50:17.680545 2126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:50:17.696291 2126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:50:17.697925 1909 heartbeater.cc:503] Master 127.7.111.126:40455 was elected leader, sending a full tablet report...
I20260811 12:50:17.701272 2126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:50:17.701561 2126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:50:17.702590 2126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:50:17.703929 2126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:50:17.704006 2126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:50:17.704120 2126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:50:17.704177 2126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:50:17.734076 2126 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:45185
I20260811 12:50:17.734124 2339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:45185 every 8 connection(s)
I20260811 12:50:17.735369 2126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:50:17.738031 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 2126
I20260811 12:50:17.738256 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:50:17.738864 7613 external_mini_cluster.cc:1428] Setting key eeff80e1e4c467c801c20084f8e2552b
Aug 11 12:50:17 dist-test-slave-f0tn krb5kdc[1521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452617, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:50:17.755414 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:17.737954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:55361 (local address 127.7.111.126:40455)
0811 12:50:17.738471 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:17.738476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:17.739669 (+ 1193us) server_negotiation.cc:407] Connection header received
0811 12:50:17.741807 (+ 2138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:17.741813 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:17.741895 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:17.742010 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:50:17.743543 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:17.744307 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:17.745865 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:17.746360 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:17.746454 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:17.749731 (+ 3277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:50:17.749736 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:50:17.749739 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:50:17.749771 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:50:17.752247 (+ 2476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:17.753035 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:17.753041 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:17.753045 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:17.753217 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:17.753740 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:17.753744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:17.753747 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:17.754065 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:50:17.754229 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:17.754971 (+ 742us) server_negotiation.cc:299] Negotiation successful
0811 12:50:17.755138 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":257,"threads_started":1}
I20260811 12:50:17.756430 2351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:17.738603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.67:55361)
0811 12:50:17.739461 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:17.739514 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 12:50:17.741536 (+ 2022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:50:17.742284 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:50:17.742293 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:50:17.742805 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:50:17.743398 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:17.743416 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:17.744427 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:17.744431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:17.745739 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:17.745752 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:17.746546 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:17.746552 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:17.746761 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:17.747395 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:50:17.747418 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:50:17.749546 (+ 2128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:50:17.752379 (+ 2833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:50:17.752387 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:50:17.752403 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:50:17.752917 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:50:17.753328 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:50:17.753333 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:50:17.753337 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:50:17.753621 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:50:17.754203 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:50:17.754209 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:50:17.754619 (+ 410us) client_negotiation.cc:769] Sending connection context
0811 12:50:17.754871 (+ 252us) client_negotiation.cc:240] Negotiation successful
0811 12:50:17.755278 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":341,"threads_started":1}
I20260811 12:50:17.757809 2340 heartbeater.cc:347] Connected to a master server at 127.7.111.126:40455
I20260811 12:50:17.758098 2340 heartbeater.cc:464] Registering TS with master...
I20260811 12:50:17.758829 2340 heartbeater.cc:511] Master 127.7.111.126:40455 requested a full tablet report, sending...
I20260811 12:50:17.760223 1589 ts_manager.cc:194] Registered new tserver with Master: d99382592a4347488d921cece4145022 (127.7.111.67:45185)
I20260811 12:50:17.760995 1589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:55361
I20260811 12:50:17.766620 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:50:17.783490 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:17.771135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33658 (local address 127.7.111.126:40455)
0811 12:50:17.771335 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:17.771342 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:17.771749 (+ 407us) server_negotiation.cc:407] Connection header received
0811 12:50:17.771888 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:17.771891 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:17.771952 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:17.772024 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:50:17.773346 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:17.774059 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:17.775407 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:17.775827 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:17.775919 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:17.778347 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:50:17.778351 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:50:17.778354 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:50:17.778384 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:50:17.780952 (+ 2568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:17.781675 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:17.781678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:17.781681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:17.781826 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:17.782114 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:17.782117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:17.782119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:17.782423 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:50:17.782608 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:17.782941 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 12:50:17.783195 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:50:17.787866 1589 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:50:17.794916 1589 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:50:17.831116 2366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:17.808258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:42023 (local address 127.0.0.1:52216)
0811 12:50:17.823261 (+ 15003us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:17.823281 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:50:17.823423 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:50:17.824460 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:50:17.824469 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:50:17.824493 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:50:17.824831 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:17.824841 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:17.826273 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:17.826279 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:17.828087 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:17.828100 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:17.830018 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:17.830025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:17.830240 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:17.830298 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 12:50:17.830420 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 12:50:17.830628 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14880,"mutex_wait_us":18,"thread_start_us":14711,"threads_started":1}
I20260811 12:50:17.831393 2367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:17.808995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39758 (local address 127.7.111.67:45185)
0811 12:50:17.822039 (+ 13044us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:17.822048 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:17.822084 (+ 36us) server_negotiation.cc:407] Connection header received
0811 12:50:17.822215 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:17.822224 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:17.822529 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:17.823525 (+ 996us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:50:17.824167 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:17.825882 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:17.828897 (+ 3015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:17.830309 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:17.830445 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:17.830665 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:17.830891 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 12:50:17.831060 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12833,"thread_start_us":12611,"threads_started":1}
I20260811 12:50:17.832026 2369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:17.809826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52216 (local address 127.7.111.66:42023)
0811 12:50:17.822360 (+ 12534us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:17.822370 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:17.823398 (+ 1028us) server_negotiation.cc:407] Connection header received
0811 12:50:17.823667 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:17.823675 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:17.823925 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:17.824098 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:50:17.824982 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:17.826097 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:17.828257 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:17.829508 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:17.829645 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:17.829863 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:17.831514 (+ 1651us) server_negotiation.cc:299] Negotiation successful
0811 12:50:17.831704 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12361,"thread_start_us":12186,"threads_started":1}
I20260811 12:50:17.831408 2364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:17.807848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:45185 (local address 127.0.0.1:39758)
0811 12:50:17.809342 (+ 1494us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:17.809410 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 12:50:17.809588 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:50:17.822837 (+ 13249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:50:17.822846 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:50:17.822886 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:50:17.823414 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:17.823425 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:17.826037 (+ 2612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:17.826042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:17.828708 (+ 2666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:17.828724 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:17.830559 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:17.830563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:17.830692 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:17.830716 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 12:50:17.830803 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 12:50:17.830930 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1243,"mutex_wait_us":100,"thread_start_us":1088,"threads_started":1}
I20260811 12:50:17.833359 2365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:17.807913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:44679 (local address 127.0.0.1:50358)
0811 12:50:17.822903 (+ 14990us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:17.822929 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 12:50:17.823098 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:50:17.823888 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:50:17.823893 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:50:17.823919 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:50:17.824397 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:17.824412 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:17.826308 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:17.826314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:17.828708 (+ 2394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:17.828724 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:17.832584 (+ 3860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:17.832591 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:17.832769 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:17.832793 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 12:50:17.832871 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 12:50:17.832994 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14843,"mutex_wait_us":12,"thread_start_us":14680,"threads_started":1}
I20260811 12:50:17.833442 2368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:17.809683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50358 (local address 127.7.111.65:44679)
0811 12:50:17.820732 (+ 11049us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:17.820743 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:17.823254 (+ 2511us) server_negotiation.cc:407] Connection header received
0811 12:50:17.823342 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:17.823349 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:17.823616 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:17.823821 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:50:17.824685 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:17.826177 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:17.830769 (+ 4592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:17.832362 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:17.832501 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:17.832719 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:17.832934 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 12:50:17.833107 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10846,"thread_start_us":10658,"threads_started":1}
I20260811 12:50:17.836149 2014 tablet_service.cc:1467] Processing CreateTablet for tablet 78977216ebbb40429eddb5c14b819717 (DEFAULT_TABLE table=test-table [id=97fadbdd488f48a0b98a6075080fb445]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:50:17.836169 2227 tablet_service.cc:1467] Processing CreateTablet for tablet 78977216ebbb40429eddb5c14b819717 (DEFAULT_TABLE table=test-table [id=97fadbdd488f48a0b98a6075080fb445]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:50:17.837807 2227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78977216ebbb40429eddb5c14b819717. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:50:17.837810 2014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78977216ebbb40429eddb5c14b819717. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:50:17.839756 1799 tablet_service.cc:1467] Processing CreateTablet for tablet 78977216ebbb40429eddb5c14b819717 (DEFAULT_TABLE table=test-table [id=97fadbdd488f48a0b98a6075080fb445]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:50:17.841174 1799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78977216ebbb40429eddb5c14b819717. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:50:17.849049 2370 tablet_bootstrap.cc:492] T 78977216ebbb40429eddb5c14b819717 P 66dcdc14b4cc4517b3b044a53d900d0d: Bootstrap starting.
I20260811 12:50:17.852180 2370 tablet_bootstrap.cc:654] T 78977216ebbb40429eddb5c14b819717 P 66dcdc14b4cc4517b3b044a53d900d0d: Neither blocks nor log segments found. Creating new log.
I20260811 12:50:17.853048 2371 tablet_bootstrap.cc:492] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022: Bootstrap starting.
I20260811 12:50:17.853575 2370 log.cc:824] T 78977216ebbb40429eddb5c14b819717 P 66dcdc14b4cc4517b3b044a53d900d0d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:50:17.856501 2371 tablet_bootstrap.cc:654] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022: Neither blocks nor log segments found. Creating new log.
I20260811 12:50:17.856987 2370 tablet_bootstrap.cc:492] T 78977216ebbb40429eddb5c14b819717 P 66dcdc14b4cc4517b3b044a53d900d0d: No bootstrap required, opened a new log
I20260811 12:50:17.857307 2370 ts_tablet_manager.cc:1397] T 78977216ebbb40429eddb5c14b819717 P 66dcdc14b4cc4517b3b044a53d900d0d: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.004s
I20260811 12:50:17.857956 2371 log.cc:824] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022: Log is configured to *not* fsync() on all Append() calls
I20260811 12:50:17.858572 2377 tablet_bootstrap.cc:492] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb: Bootstrap starting.
I20260811 12:50:17.861665 2377 tablet_bootstrap.cc:654] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb: Neither blocks nor log segments found. Creating new log.
I20260811 12:50:17.862922 2377 log.cc:824] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb: Log is configured to *not* fsync() on all Append() calls
I20260811 12:50:17.864252 2371 tablet_bootstrap.cc:492] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022: No bootstrap required, opened a new log
I20260811 12:50:17.864528 2371 ts_tablet_manager.cc:1397] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.006s
I20260811 12:50:17.865622 2377 tablet_bootstrap.cc:492] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb: No bootstrap required, opened a new log
I20260811 12:50:17.865865 2377 ts_tablet_manager.cc:1397] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 12:50:17.867545 2370 raft_consensus.cc:348] T 78977216ebbb40429eddb5c14b819717 P 66dcdc14b4cc4517b3b044a53d900d0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99382592a4347488d921cece4145022" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45185 } } peers { permanent_uuid: "c2766f606ebd4d81a30334721da429eb" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 44679 } } peers { permanent_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42023 } }
I20260811 12:50:17.868026 2370 raft_consensus.cc:374] T 78977216ebbb40429eddb5c14b819717 P 66dcdc14b4cc4517b3b044a53d900d0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:50:17.868151 2370 raft_consensus.cc:729] T 78977216ebbb40429eddb5c14b819717 P 66dcdc14b4cc4517b3b044a53d900d0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66dcdc14b4cc4517b3b044a53d900d0d, State: Initialized, Role: FOLLOWER
I20260811 12:50:17.868788 2370 consensus_queue.cc:260] T 78977216ebbb40429eddb5c14b819717 P 66dcdc14b4cc4517b3b044a53d900d0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99382592a4347488d921cece4145022" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45185 } } peers { permanent_uuid: "c2766f606ebd4d81a30334721da429eb" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 44679 } } peers { permanent_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42023 } }
I20260811 12:50:17.874188 2113 heartbeater.cc:503] Master 127.7.111.126:40455 was elected leader, sending a full tablet report...
I20260811 12:50:17.874797 2370 ts_tablet_manager.cc:1428] T 78977216ebbb40429eddb5c14b819717 P 66dcdc14b4cc4517b3b044a53d900d0d: Time spent starting tablet: real 0.017s user 0.010s sys 0.006s
I20260811 12:50:17.875126 2371 raft_consensus.cc:348] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99382592a4347488d921cece4145022" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45185 } } peers { permanent_uuid: "c2766f606ebd4d81a30334721da429eb" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 44679 } } peers { permanent_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42023 } }
I20260811 12:50:17.875322 2377 raft_consensus.cc:348] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99382592a4347488d921cece4145022" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45185 } } peers { permanent_uuid: "c2766f606ebd4d81a30334721da429eb" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 44679 } } peers { permanent_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42023 } }
I20260811 12:50:17.875572 2371 raft_consensus.cc:374] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:50:17.875669 2371 raft_consensus.cc:729] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d99382592a4347488d921cece4145022, State: Initialized, Role: FOLLOWER
I20260811 12:50:17.875699 2377 raft_consensus.cc:374] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:50:17.875785 2377 raft_consensus.cc:729] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2766f606ebd4d81a30334721da429eb, State: Initialized, Role: FOLLOWER
I20260811 12:50:17.876279 2371 consensus_queue.cc:260] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99382592a4347488d921cece4145022" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45185 } } peers { permanent_uuid: "c2766f606ebd4d81a30334721da429eb" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 44679 } } peers { permanent_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42023 } }
I20260811 12:50:17.876410 2377 consensus_queue.cc:260] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99382592a4347488d921cece4145022" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45185 } } peers { permanent_uuid: "c2766f606ebd4d81a30334721da429eb" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 44679 } } peers { permanent_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42023 } }
I20260811 12:50:17.877467 2340 heartbeater.cc:503] Master 127.7.111.126:40455 was elected leader, sending a full tablet report...
I20260811 12:50:17.878571 2377 ts_tablet_manager.cc:1428] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb: Time spent starting tablet: real 0.013s user 0.010s sys 0.001s
I20260811 12:50:17.879362 2371 ts_tablet_manager.cc:1428] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022: Time spent starting tablet: real 0.015s user 0.013s sys 0.002s
W20260811 12:50:17.920607 1911 tablet.cc:2369] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:50:17.948156 2115 tablet.cc:2369] T 78977216ebbb40429eddb5c14b819717 P 66dcdc14b4cc4517b3b044a53d900d0d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:50:17.986922 2344 tablet.cc:2369] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:50:18.045023 2384 raft_consensus.cc:482] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:50:18.045359 2384 raft_consensus.cc:504] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99382592a4347488d921cece4145022" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45185 } } peers { permanent_uuid: "c2766f606ebd4d81a30334721da429eb" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 44679 } } peers { permanent_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42023 } }
I20260811 12:50:18.047231 2384 leader_election.cc:288] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2766f606ebd4d81a30334721da429eb (127.7.111.65:44679), 66dcdc14b4cc4517b3b044a53d900d0d (127.7.111.66:42023)
I20260811 12:50:18.053076 2385 raft_consensus.cc:482] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:50:18.053228 2351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:18.047456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:44679 (local address 127.7.111.67:47931)
0811 12:50:18.047837 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:18.047859 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:50:18.047995 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:50:18.048396 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:50:18.048401 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:50:18.048431 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:50:18.048648 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:18.048660 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.049950 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.049955 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:18.051406 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:18.051419 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.052470 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.052476 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:18.052657 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:18.052690 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:50:18.052780 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 12:50:18.052920 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":1}
I20260811 12:50:18.053593 2368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:18.047652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:47931 (local address 127.7.111.65:44679)
0811 12:50:18.047865 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:18.047870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:18.047941 (+ 71us) server_negotiation.cc:407] Connection header received
0811 12:50:18.048152 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:18.048157 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:18.048250 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:18.048349 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:50:18.048781 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.049815 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.051558 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.052321 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.052465 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:18.052549 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:18.052941 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 12:50:18.053077 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 12:50:18.053655 2369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:18.047659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:60061 (local address 127.7.111.66:42023)
0811 12:50:18.047869 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:18.047875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:18.048513 (+ 638us) server_negotiation.cc:407] Connection header received
0811 12:50:18.048649 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:18.048653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:18.048722 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:18.048807 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:50:18.049286 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.050190 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.051904 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.052643 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.052815 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:18.052886 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:18.053259 (+ 373us) server_negotiation.cc:299] Negotiation successful
0811 12:50:18.053412 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 12:50:18.053427 2385 raft_consensus.cc:504] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99382592a4347488d921cece4145022" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45185 } } peers { permanent_uuid: "c2766f606ebd4d81a30334721da429eb" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 44679 } } peers { permanent_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42023 } }
I20260811 12:50:18.054311 2388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:18.047641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:42023 (local address 127.7.111.67:60061)
0811 12:50:18.048400 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:18.048418 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:50:18.048535 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:50:18.048826 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:50:18.048829 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:50:18.048849 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:50:18.049139 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:18.049146 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.050339 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.050343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:18.051752 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:18.051766 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.052866 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.052872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:18.053038 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:18.053076 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 12:50:18.053870 (+ 794us) client_negotiation.cc:240] Negotiation successful
0811 12:50:18.053994 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":427,"threads_started":1}
I20260811 12:50:18.054487 1835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78977216ebbb40429eddb5c14b819717" candidate_uuid: "d99382592a4347488d921cece4145022" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2766f606ebd4d81a30334721da429eb" is_pre_election: true
I20260811 12:50:18.054916 1835 raft_consensus.cc:2454] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d99382592a4347488d921cece4145022 in term 0.
I20260811 12:50:18.055276 2039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78977216ebbb40429eddb5c14b819717" candidate_uuid: "d99382592a4347488d921cece4145022" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" is_pre_election: true
I20260811 12:50:18.055575 2150 leader_election.cc:302] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c2766f606ebd4d81a30334721da429eb, d99382592a4347488d921cece4145022; no voters:
I20260811 12:50:18.055843 2039 raft_consensus.cc:2454] T 78977216ebbb40429eddb5c14b819717 P 66dcdc14b4cc4517b3b044a53d900d0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d99382592a4347488d921cece4145022 in term 0.
I20260811 12:50:18.055965 2384 raft_consensus.cc:2781] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:50:18.056087 2384 raft_consensus.cc:482] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:50:18.056169 2384 raft_consensus.cc:3037] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:50:18.058054 2385 leader_election.cc:288] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d99382592a4347488d921cece4145022 (127.7.111.67:45185), 66dcdc14b4cc4517b3b044a53d900d0d (127.7.111.66:42023)
I20260811 12:50:18.059087 2384 raft_consensus.cc:504] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99382592a4347488d921cece4145022" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45185 } } peers { permanent_uuid: "c2766f606ebd4d81a30334721da429eb" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 44679 } } peers { permanent_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42023 } }
I20260811 12:50:18.060154 2384 leader_election.cc:288] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022 [CANDIDATE]: Term 1 election: Requested vote from peers c2766f606ebd4d81a30334721da429eb (127.7.111.65:44679), 66dcdc14b4cc4517b3b044a53d900d0d (127.7.111.66:42023)
I20260811 12:50:18.060283 2039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78977216ebbb40429eddb5c14b819717" candidate_uuid: "d99382592a4347488d921cece4145022" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66dcdc14b4cc4517b3b044a53d900d0d"
I20260811 12:50:18.060474 2039 raft_consensus.cc:3037] T 78977216ebbb40429eddb5c14b819717 P 66dcdc14b4cc4517b3b044a53d900d0d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:50:18.060500 1835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78977216ebbb40429eddb5c14b819717" candidate_uuid: "d99382592a4347488d921cece4145022" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2766f606ebd4d81a30334721da429eb"
I20260811 12:50:18.060714 1835 raft_consensus.cc:3037] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:50:18.063869 2039 raft_consensus.cc:2454] T 78977216ebbb40429eddb5c14b819717 P 66dcdc14b4cc4517b3b044a53d900d0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d99382592a4347488d921cece4145022 in term 1.
I20260811 12:50:18.064083 1835 raft_consensus.cc:2454] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d99382592a4347488d921cece4145022 in term 1.
I20260811 12:50:18.064324 2152 leader_election.cc:302] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66dcdc14b4cc4517b3b044a53d900d0d, d99382592a4347488d921cece4145022; no voters:
I20260811 12:50:18.064860 2384 raft_consensus.cc:2781] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:50:18.065322 2384 raft_consensus.cc:686] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022 [term 1 LEADER]: Becoming Leader. State: Replica: d99382592a4347488d921cece4145022, State: Running, Role: LEADER
I20260811 12:50:18.065866 2384 consensus_queue.cc:237] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99382592a4347488d921cece4145022" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45185 } } peers { permanent_uuid: "c2766f606ebd4d81a30334721da429eb" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 44679 } } peers { permanent_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42023 } }
I20260811 12:50:18.066578 2369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:18.058456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:47755 (local address 127.7.111.66:42023)
0811 12:50:18.058668 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:18.058673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:18.059625 (+ 952us) server_negotiation.cc:407] Connection header received
0811 12:50:18.059811 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:18.059816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:18.059885 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:18.059942 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:50:18.060769 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.061970 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.064726 (+ 2756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.065573 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.065715 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:18.065785 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:18.066199 (+ 414us) server_negotiation.cc:299] Negotiation successful
0811 12:50:18.066325 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":103}
I20260811 12:50:18.066841 2390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:18.058368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:42023 (local address 127.7.111.65:47755)
0811 12:50:18.059533 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:18.059549 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:50:18.059657 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:50:18.060124 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:50:18.060129 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:50:18.060156 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:50:18.060516 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:18.060529 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.062150 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.062154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:18.064571 (+ 2417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:18.064586 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.065785 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.065791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:18.065951 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:18.065985 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 12:50:18.066176 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 12:50:18.066305 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1028,"mutex_wait_us":80,"thread_start_us":854,"threads_started":1}
I20260811 12:50:18.067354 2039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78977216ebbb40429eddb5c14b819717" candidate_uuid: "c2766f606ebd4d81a30334721da429eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" is_pre_election: true
I20260811 12:50:18.067629 2039 raft_consensus.cc:2379] T 78977216ebbb40429eddb5c14b819717 P 66dcdc14b4cc4517b3b044a53d900d0d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c2766f606ebd4d81a30334721da429eb in current term 1: Already voted for candidate d99382592a4347488d921cece4145022 in this term.
I20260811 12:50:18.067986 2367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:18.059949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:58735 (local address 127.7.111.67:45185)
0811 12:50:18.060121 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:18.060126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:18.060148 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:50:18.060199 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:18.060203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:18.060268 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:18.060350 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:50:18.061160 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.062192 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.065870 (+ 3678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.067096 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.067247 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:18.067312 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:18.067613 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 12:50:18.067763 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 12:50:18.068107 2389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:18.056065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:45185 (local address 127.7.111.65:58735)
0811 12:50:18.059235 (+ 3170us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:18.059254 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:50:18.059434 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:50:18.060502 (+ 1068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:50:18.060506 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:50:18.060528 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:50:18.060982 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:18.061000 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.062379 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.062384 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:18.064574 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:18.064589 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.067301 (+ 2712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.067307 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:18.067461 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:18.067491 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 12:50:18.067580 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 12:50:18.067763 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3022,"mutex_wait_us":5,"thread_start_us":2829,"threads_started":1}
I20260811 12:50:18.069134 2264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78977216ebbb40429eddb5c14b819717" candidate_uuid: "c2766f606ebd4d81a30334721da429eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d99382592a4347488d921cece4145022" is_pre_election: true
I20260811 12:50:18.070197 1725 leader_election.cc:302] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c2766f606ebd4d81a30334721da429eb; no voters: 66dcdc14b4cc4517b3b044a53d900d0d, d99382592a4347488d921cece4145022
I20260811 12:50:18.070525 2385 raft_consensus.cc:2726] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 12:50:18.071933 1589 catalog_manager.cc:5495] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022 reported cstate change: term changed from 0 to 1, leader changed from <none> to d99382592a4347488d921cece4145022 (127.7.111.67). New cstate: current_term: 1 leader_uuid: "d99382592a4347488d921cece4145022" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99382592a4347488d921cece4145022" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2766f606ebd4d81a30334721da429eb" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42023 } health_report { overall_health: UNKNOWN } } }
I20260811 12:50:18.111531 2367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:18.106627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39770 (local address 127.7.111.67:45185)
0811 12:50:18.106790 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:18.106794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:18.106876 (+ 82us) server_negotiation.cc:407] Connection header received
0811 12:50:18.107049 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:18.107053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:18.107134 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:18.107260 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:50:18.107605 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.108355 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.109641 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.110043 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.110130 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:18.110580 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:50:18.111091 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:50:18.111208 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:18.111245 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 12:50:18.111305 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 12:50:18.119927 2039 raft_consensus.cc:1261] T 78977216ebbb40429eddb5c14b819717 P 66dcdc14b4cc4517b3b044a53d900d0d [term 1 FOLLOWER]: Refusing update from remote peer d99382592a4347488d921cece4145022: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:50:18.119975 1835 raft_consensus.cc:1261] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb [term 1 FOLLOWER]: Refusing update from remote peer d99382592a4347488d921cece4145022: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:50:18.120939 2391 consensus_queue.cc:1035] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:50:18.121294 2384 consensus_queue.cc:1035] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2766f606ebd4d81a30334721da429eb" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:50:18.151732 1589 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:50:18.151979 1589 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:50:18.155658 1589 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb: Sending DeleteTablet for 3 replicas of tablet 78977216ebbb40429eddb5c14b819717
I20260811 12:50:18.156872 2227 tablet_service.cc:1514] Processing DeleteTablet for tablet 78977216ebbb40429eddb5c14b819717 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:50:18 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:39758
I20260811 12:50:18.156905 1799 tablet_service.cc:1514] Processing DeleteTablet for tablet 78977216ebbb40429eddb5c14b819717 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:50:18 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:50358
I20260811 12:50:18.157629 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 1696
I20260811 12:50:18.157656 2014 tablet_service.cc:1514] Processing DeleteTablet for tablet 78977216ebbb40429eddb5c14b819717 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:50:18 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:52216
I20260811 12:50:18.165925 2413 tablet_replica.cc:330] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb: stopping tablet replica
I20260811 12:50:18.166211 2417 tablet_replica.cc:330] T 78977216ebbb40429eddb5c14b819717 P 66dcdc14b4cc4517b3b044a53d900d0d: stopping tablet replica
I20260811 12:50:18.166432 2413 raft_consensus.cc:2229] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:50:18.166719 2417 raft_consensus.cc:2229] T 78977216ebbb40429eddb5c14b819717 P 66dcdc14b4cc4517b3b044a53d900d0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:50:18.167001 2413 raft_consensus.cc:2258] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:50:18.167330 2417 raft_consensus.cc:2258] T 78977216ebbb40429eddb5c14b819717 P 66dcdc14b4cc4517b3b044a53d900d0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:50:18.167768 2416 tablet_replica.cc:330] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022: stopping tablet replica
I20260811 12:50:18.169843 2417 ts_tablet_manager.cc:1905] T 78977216ebbb40429eddb5c14b819717 P 66dcdc14b4cc4517b3b044a53d900d0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:50:18.169898 2413 ts_tablet_manager.cc:1905] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:50:18.171636 2416 raft_consensus.cc:2229] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:50:18.172402 2416 raft_consensus.cc:2258] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:50:18.174546 2416 ts_tablet_manager.cc:1905] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:50:18.178664 2413 ts_tablet_manager.cc:1918] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:50:18.178831 2413 log.cc:1196] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/78977216ebbb40429eddb5c14b819717
I20260811 12:50:18.179209 2417 ts_tablet_manager.cc:1918] T 78977216ebbb40429eddb5c14b819717 P 66dcdc14b4cc4517b3b044a53d900d0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:50:18.179394 2413 ts_tablet_manager.cc:1939] T 78977216ebbb40429eddb5c14b819717 P c2766f606ebd4d81a30334721da429eb: Deleting consensus metadata
I20260811 12:50:18.179416 2417 log.cc:1196] T 78977216ebbb40429eddb5c14b819717 P 66dcdc14b4cc4517b3b044a53d900d0d: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/78977216ebbb40429eddb5c14b819717
I20260811 12:50:18.179981 2417 ts_tablet_manager.cc:1939] T 78977216ebbb40429eddb5c14b819717 P 66dcdc14b4cc4517b3b044a53d900d0d: Deleting consensus metadata
I20260811 12:50:18.181301 1566 catalog_manager.cc:4841] TS c2766f606ebd4d81a30334721da429eb (127.7.111.65:44679): tablet 78977216ebbb40429eddb5c14b819717 (table test-table [id=97fadbdd488f48a0b98a6075080fb445]) successfully deleted
I20260811 12:50:18.181792 1568 catalog_manager.cc:4841] TS 66dcdc14b4cc4517b3b044a53d900d0d (127.7.111.66:42023): tablet 78977216ebbb40429eddb5c14b819717 (table test-table [id=97fadbdd488f48a0b98a6075080fb445]) successfully deleted
I20260811 12:50:18.184583 2416 ts_tablet_manager.cc:1918] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:50:18.184731 2416 log.cc:1196] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/78977216ebbb40429eddb5c14b819717
I20260811 12:50:18.185159 2416 ts_tablet_manager.cc:1939] T 78977216ebbb40429eddb5c14b819717 P d99382592a4347488d921cece4145022: Deleting consensus metadata
Aug 11 12:50:18 dist-test-slave-f0tn krb5kdc[1521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.65@KRBTEST.COM
I20260811 12:50:18.186563 1569 catalog_manager.cc:4841] TS d99382592a4347488d921cece4145022 (127.7.111.67:45185): tablet 78977216ebbb40429eddb5c14b819717 (table test-table [id=97fadbdd488f48a0b98a6075080fb445]) successfully deleted
I20260811 12:50:18.192330 2368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:18.158748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50360 (local address 127.7.111.65:44679)
0811 12:50:18.158922 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:18.158929 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:18.168273 (+ 9344us) server_negotiation.cc:407] Connection header received
0811 12:50:18.168600 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:18.168605 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:18.168702 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:18.168779 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:50:18.171361 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.172430 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.174498 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.175086 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.175201 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:18.186784 (+ 11583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:50:18.186797 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:50:18.186812 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:50:18.186853 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:50:18.189642 (+ 2789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:18.190468 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:18.190475 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:18.190477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:18.190602 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:18.190976 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:18.190979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:18.190981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:18.191327 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:50:18.191556 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:18.191894 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 12:50:18.192064 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:50:18.192958 1900 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:50:18.294869 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 1696
I20260811 12:50:18.336923 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 1919
Aug 11 12:50:18 dist-test-slave-f0tn krb5kdc[1521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.66@KRBTEST.COM
I20260811 12:50:18.353634 2369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:18.337802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52230 (local address 127.7.111.66:42023)
0811 12:50:18.338021 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:18.338027 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:18.338049 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:50:18.338170 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:18.338176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:18.338269 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:18.338376 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:50:18.339593 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.340682 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.342190 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.342852 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.342982 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:18.347804 (+ 4822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:50:18.347820 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:50:18.347839 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:50:18.347881 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:50:18.350805 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:18.351647 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:18.351654 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:18.351657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:18.351792 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:18.352188 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:18.352191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:18.352193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:18.352555 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:50:18.352801 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:18.353146 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 12:50:18.353325 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 12:50:18.354410 2105 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:50:18.443353 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 1919
I20260811 12:50:18.477305 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 2126
Aug 11 12:50:18 dist-test-slave-f0tn krb5kdc[1521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.67@KRBTEST.COM
I20260811 12:50:18.493731 2367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:18.478142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39778 (local address 127.7.111.67:45185)
0811 12:50:18.478385 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:18.478391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:18.478412 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:50:18.478545 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:18.478551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:18.478632 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:18.478719 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:50:18.480105 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.480882 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.482247 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:18.482732 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:18.482859 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:18.487719 (+ 4860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:50:18.487733 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:50:18.487751 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:50:18.487790 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:50:18.490889 (+ 3099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:18.491563 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:18.491571 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:18.491573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:18.491735 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:18.492101 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:18.492105 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:18.492107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:18.492529 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:50:18.492864 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:18.493145 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 12:50:18.493404 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 12:50:18.494473 2333 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:50:18.621956 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 2126
I20260811 12:50:18.651504 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 1545
I20260811 12:50:18.652702 1652 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:50:18.764415 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 1545
I20260811 12:50:38.795696 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:40455
--webserver_interface=127.7.111.126
--webserver_port=45933
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:44687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:40455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:50:39.139073 2424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:50:39.139395 2424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:50:39.139531 2424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:50:39.146898 2424 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 12:50:39.147003 2424 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 12:50:39.147042 2424 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 12:50:39.147115 2424 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:50:39.147197 2424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:50:39.147301 2424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:50:39.147361 2424 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:50:39.147439 2424 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:50:39.157509 2424 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:44687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:40455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:40455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=45933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:50:39.158851 2424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:50:39.160271 2424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:50:39.184912 2436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:50:39.184903 2438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:50:39.185043 2434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:50:39.185575 2424 server_base.cc:1034] running on GCE node
I20260811 12:50:39.186515 2424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:50:39.188126 2424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:50:39.189361 2424 hybrid_clock.cc:648] HybridClock initialized: now 1786452639189296 us; error 36 us; skew 500 ppm
I20260811 12:50:39.189786 2424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:50:39 dist-test-slave-f0tn krb5kdc[1521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452639, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:50:39.197057 2424 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:50:39.198884 2424 webserver.cc:460] Webserver started at http://127.7.111.126:45933/ using document root <none> and password file <none>
I20260811 12:50:39.199685 2424 fs_manager.cc:359] Metadata directory not provided
I20260811 12:50:39.199813 2424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:50:39.205971 2424 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 12:50:39.208856 2452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:50:39.210268 2424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:50:39.210443 2424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "17ccc579f9934760a2dcd371e1c026fb"
format_stamp: "Formatted at 2026-08-11 12:50:15 on dist-test-slave-f0tn"
server_key: "7183e2e070a64cf53f27eb77929542e5"
server_key_iv: "4aad647f02b61c7b03459e2a3b25302e"
server_key_version: "encryptionkey@0"
I20260811 12:50:39.211532 2424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:50:39.231138 2424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:50:39.235599 2424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:50:39.235899 2424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:50:39.254367 2424 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:40455
I20260811 12:50:39.254397 2536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:40455 every 8 connection(s)
I20260811 12:50:39.255926 2424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:50:39.258577 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 2424
I20260811 12:50:39.259443 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:44679
--local_ip_for_outbound_sockets=127.7.111.65
--tserver_master_addrs=127.7.111.126:40455
--webserver_port=34863
--webserver_interface=127.7.111.65
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:44687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:50:39.263402 2538 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:50:39.266881 2538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb: Bootstrap starting.
I20260811 12:50:39.287235 2538 log.cc:824] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb: Log is configured to *not* fsync() on all Append() calls
I20260811 12:50:39.299638 2538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:50:39.300170 2538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb: Bootstrap complete.
I20260811 12:50:39.309264 2538 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ccc579f9934760a2dcd371e1c026fb" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40455 } }
I20260811 12:50:39.310216 2538 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17ccc579f9934760a2dcd371e1c026fb, State: Initialized, Role: FOLLOWER
I20260811 12:50:39.310817 2538 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ccc579f9934760a2dcd371e1c026fb" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40455 } }
I20260811 12:50:39.311056 2538 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:50:39.311168 2538 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:50:39.311338 2538 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 1 FOLLOWER]: Advancing to term 2
I20260811 12:50:39.315220 2538 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ccc579f9934760a2dcd371e1c026fb" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40455 } }
I20260811 12:50:39.315748 2538 leader_election.cc:302] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17ccc579f9934760a2dcd371e1c026fb; no voters:
I20260811 12:50:39.316442 2538 leader_election.cc:288] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 12:50:39.316618 2547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 2 FOLLOWER]: Leader election won for term 2
I20260811 12:50:39.317684 2547 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 2 LEADER]: Becoming Leader. State: Replica: 17ccc579f9934760a2dcd371e1c026fb, State: Running, Role: LEADER
I20260811 12:50:39.318413 2547 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ccc579f9934760a2dcd371e1c026fb" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40455 } }
I20260811 12:50:39.318578 2538 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:50:39.322477 2550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17ccc579f9934760a2dcd371e1c026fb. Latest consensus state: current_term: 2 leader_uuid: "17ccc579f9934760a2dcd371e1c026fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ccc579f9934760a2dcd371e1c026fb" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40455 } } }
I20260811 12:50:39.322871 2550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [sys.catalog]: This master's current role is: LEADER
I20260811 12:50:39.323324 2548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "17ccc579f9934760a2dcd371e1c026fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ccc579f9934760a2dcd371e1c026fb" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40455 } } }
I20260811 12:50:39.323625 2548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [sys.catalog]: This master's current role is: LEADER
I20260811 12:50:39.335374 2559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:50:39.343207 2559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:50:39.344720 2559 catalog_manager.cc:1259] Loaded cluster ID: f96b0f04889143ec87815c16db24b363
I20260811 12:50:39.344954 2559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:50:39.349068 2559 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:50:39.350647 2559 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb: Loaded TSK: 0
I20260811 12:50:39.395455 2559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb: Generated new TSK 1
I20260811 12:50:39.395850 2559 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 12:50:39.644506 2542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:50:39.644817 2542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:50:39.644936 2542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:50:39.652369 2542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:50:39.652485 2542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:50:39.652622 2542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:50:39.662539 2542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:44687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:44679
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=34863
--tserver_master_addrs=127.7.111.126:40455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:50:39.663848 2542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:50:39.665198 2542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:50:39.679173 2577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:50:39.679504 2583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:50:39.679668 2581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:50:39.680030 2542 server_base.cc:1034] running on GCE node
I20260811 12:50:39.680927 2542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:50:39.682353 2542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:50:39.683630 2542 hybrid_clock.cc:648] HybridClock initialized: now 1786452639683565 us; error 46 us; skew 500 ppm
I20260811 12:50:39.684077 2542 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:50:39 dist-test-slave-f0tn krb5kdc[1521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452639, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:50:39.688467 2542 init.cc:377] Logged in from keytab as kudu/127.7.111.65@KRBTEST.COM (short username kudu)
I20260811 12:50:39.690508 2542 webserver.cc:460] Webserver started at http://127.7.111.65:34863/ using document root <none> and password file <none>
I20260811 12:50:39.691536 2542 fs_manager.cc:359] Metadata directory not provided
I20260811 12:50:39.691721 2542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:50:39.699338 2542 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:50:39.703066 2593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:50:39.704763 2542 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:50:39.704965 2542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "c2766f606ebd4d81a30334721da429eb"
format_stamp: "Formatted at 2026-08-11 12:50:16 on dist-test-slave-f0tn"
server_key: "d98424a089994169da8f3065415057f6"
server_key_iv: "9a4105b73c3f2ef0ec114fab74f642ea"
server_key_version: "encryptionkey@0"
I20260811 12:50:39.706462 2542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:50:39.727669 2542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:50:39.732916 2542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:50:39.733273 2542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:50:39.734587 2542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:50:39.736402 2542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:50:39.736522 2542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:50:39.736639 2542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:50:39.736724 2542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:50:39.770007 2542 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:44679
I20260811 12:50:39.770030 2769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:44679 every 8 connection(s)
I20260811 12:50:39.771816 2542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:50:39.782034 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 2542
I20260811 12:50:39.782645 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:42023
--local_ip_for_outbound_sockets=127.7.111.66
--tserver_master_addrs=127.7.111.126:40455
--webserver_port=34271
--webserver_interface=127.7.111.66
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:44687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:50:39 dist-test-slave-f0tn krb5kdc[1521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452639, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:50:39.795640 2776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:39.775041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.65:48245)
0811 12:50:39.775801 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:39.775837 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 12:50:39.777550 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:50:39.778903 (+ 1353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:50:39.778914 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:50:39.779515 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:50:39.780153 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:39.780170 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:39.781949 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:39.781953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:39.783298 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:39.783311 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:39.784618 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:39.784624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:39.784858 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:39.785566 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:50:39.785588 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:50:39.787865 (+ 2277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:50:39.791783 (+ 3918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:50:39.791790 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:50:39.791801 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:50:39.792217 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:50:39.792644 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:50:39.792647 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:50:39.792649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:50:39.792821 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:50:39.793674 (+ 853us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:50:39.793681 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:50:39.793965 (+ 284us) client_negotiation.cc:769] Sending connection context
0811 12:50:39.794207 (+ 242us) client_negotiation.cc:240] Negotiation successful
0811 12:50:39.794579 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":314,"threads_started":1}
I20260811 12:50:39.795883 2773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:39.774765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:48245 (local address 127.7.111.126:40455)
0811 12:50:39.775407 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:39.775424 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:39.775969 (+ 545us) server_negotiation.cc:407] Connection header received
0811 12:50:39.778074 (+ 2105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:39.778087 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:39.778407 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:39.778723 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:50:39.780378 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:39.781777 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:39.783441 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:39.784393 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:39.784570 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:39.788137 (+ 3567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:50:39.788147 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:50:39.788166 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:50:39.788226 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:50:39.791629 (+ 3403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:39.792341 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:39.792349 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:39.792357 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:39.792554 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:39.792914 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:39.792918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:39.792920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:39.793553 (+ 633us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:50:39.793731 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:39.794433 (+ 702us) server_negotiation.cc:299] Negotiation successful
0811 12:50:39.794816 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":268,"threads_started":1}
I20260811 12:50:39.799413 2770 heartbeater.cc:347] Connected to a master server at 127.7.111.126:40455
I20260811 12:50:39.799713 2770 heartbeater.cc:464] Registering TS with master...
I20260811 12:50:39.800423 2770 heartbeater.cc:511] Master 127.7.111.126:40455 requested a full tablet report, sending...
I20260811 12:50:39.802860 2474 ts_manager.cc:194] Registered new tserver with Master: c2766f606ebd4d81a30334721da429eb (127.7.111.65:44679)
I20260811 12:50:39.804849 2474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:48245
W20260811 12:50:40.129292 2780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:50:40.129616 2780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:50:40.129727 2780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:50:40.136788 2780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:50:40.136881 2780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:50:40.136997 2780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:50:40.146575 2780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:44687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:42023
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=34271
--tserver_master_addrs=127.7.111.126:40455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:50:40.147850 2780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:50:40.149163 2780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:50:40.161808 2791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:50:40.161855 2785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:50:40.161880 2786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:50:40.162575 2780 server_base.cc:1034] running on GCE node
I20260811 12:50:40.163125 2780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:50:40.164000 2780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:50:40.165194 2780 hybrid_clock.cc:648] HybridClock initialized: now 1786452640165148 us; error 35 us; skew 500 ppm
I20260811 12:50:40.165531 2780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:50:40 dist-test-slave-f0tn krb5kdc[1521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452640, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:50:40.168506 2780 init.cc:377] Logged in from keytab as kudu/127.7.111.66@KRBTEST.COM (short username kudu)
I20260811 12:50:40.169875 2780 webserver.cc:460] Webserver started at http://127.7.111.66:34271/ using document root <none> and password file <none>
I20260811 12:50:40.170493 2780 fs_manager.cc:359] Metadata directory not provided
I20260811 12:50:40.170603 2780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:50:40.175716 2780 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 12:50:40.178355 2800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:50:40.179596 2780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:50:40.179752 2780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "66dcdc14b4cc4517b3b044a53d900d0d"
format_stamp: "Formatted at 2026-08-11 12:50:17 on dist-test-slave-f0tn"
server_key: "033a81ef562d2557a07e95f58152f6f5"
server_key_iv: "cccf2fa4f344d0e2f7f390517e0c37f0"
server_key_version: "encryptionkey@0"
I20260811 12:50:40.180779 2780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:50:40.200491 2780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:50:40.204276 2780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:50:40.204511 2780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:50:40.205474 2780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:50:40.206811 2780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:50:40.206897 2780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:50:40.207005 2780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:50:40.207072 2780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:50:40.238184 2780 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:42023
I20260811 12:50:40.238253 2975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:42023 every 8 connection(s)
I20260811 12:50:40.239574 2780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:50:40.246837 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 2780
I20260811 12:50:40.247560 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:45185
--local_ip_for_outbound_sockets=127.7.111.67
--tserver_master_addrs=127.7.111.126:40455
--webserver_port=33979
--webserver_interface=127.7.111.67
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:44687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:50:40 dist-test-slave-f0tn krb5kdc[1521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452640, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:50:40.260023 2773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:40.242160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:40547 (local address 127.7.111.126:40455)
0811 12:50:40.242348 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:40.242352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:40.243520 (+ 1168us) server_negotiation.cc:407] Connection header received
0811 12:50:40.245340 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:40.245345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:40.245468 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:40.245555 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:50:40.247024 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.248013 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.249555 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.250125 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.250258 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:40.253441 (+ 3183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:50:40.253447 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:50:40.253452 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:50:40.253490 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:50:40.256740 (+ 3250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:40.257613 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:40.257619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:40.257622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:40.257767 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:40.258209 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:40.258214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:40.258217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:40.258578 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:50:40.258719 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:40.259485 (+ 766us) server_negotiation.cc:299] Negotiation successful
0811 12:50:40.259735 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 12:50:40.261106 2985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:40.242630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.66:40547)
0811 12:50:40.243344 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:40.243393 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 12:50:40.245078 (+ 1685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:50:40.245812 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:50:40.245821 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:50:40.246299 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:50:40.246838 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:40.246855 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.248171 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.248175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:40.249369 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:40.249415 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.250322 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.250326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:40.250524 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:40.251124 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:50:40.251151 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:50:40.253197 (+ 2046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:50:40.256880 (+ 3683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:50:40.256889 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:50:40.256903 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:50:40.257488 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:50:40.257868 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:50:40.257873 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:50:40.257877 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:50:40.258095 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:50:40.258699 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:50:40.258714 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:50:40.259095 (+ 381us) client_negotiation.cc:769] Sending connection context
0811 12:50:40.259386 (+ 291us) client_negotiation.cc:240] Negotiation successful
0811 12:50:40.259892 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":266,"threads_started":1}
I20260811 12:50:40.262686 2977 heartbeater.cc:347] Connected to a master server at 127.7.111.126:40455
I20260811 12:50:40.262969 2977 heartbeater.cc:464] Registering TS with master...
I20260811 12:50:40.263803 2977 heartbeater.cc:511] Master 127.7.111.126:40455 requested a full tablet report, sending...
I20260811 12:50:40.265357 2474 ts_manager.cc:194] Registered new tserver with Master: 66dcdc14b4cc4517b3b044a53d900d0d (127.7.111.66:42023)
I20260811 12:50:40.266496 2474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:40547
W20260811 12:50:40.596123 2988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:50:40.596457 2988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:50:40.596597 2988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:50:40.604148 2988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:50:40.604295 2988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:50:40.604452 2988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:50:40.614512 2988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:44687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:45185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=33979
--tserver_master_addrs=127.7.111.126:40455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:50:40.615957 2988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:50:40.617357 2988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:50:40.630450 2993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:50:40.630473 2996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:50:40.630735 2998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:50:40.631796 2988 server_base.cc:1034] running on GCE node
I20260811 12:50:40.632457 2988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:50:40.633455 2988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:50:40.634663 2988 hybrid_clock.cc:648] HybridClock initialized: now 1786452640634597 us; error 58 us; skew 500 ppm
I20260811 12:50:40.634976 2988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:50:40 dist-test-slave-f0tn krb5kdc[1521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452640, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:50:40.638051 2988 init.cc:377] Logged in from keytab as kudu/127.7.111.67@KRBTEST.COM (short username kudu)
I20260811 12:50:40.639431 2988 webserver.cc:460] Webserver started at http://127.7.111.67:33979/ using document root <none> and password file <none>
I20260811 12:50:40.640105 2988 fs_manager.cc:359] Metadata directory not provided
I20260811 12:50:40.640223 2988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:50:40.645747 2988 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260811 12:50:40.648499 3010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:50:40.649938 2988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:50:40.650090 2988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "d99382592a4347488d921cece4145022"
format_stamp: "Formatted at 2026-08-11 12:50:17 on dist-test-slave-f0tn"
server_key: "c4d5aacbceee4de22be82aaed2c87f01"
server_key_iv: "a3e8a8e6e6ecc2a199b9724cd9d36c45"
server_key_version: "encryptionkey@0"
I20260811 12:50:40.651085 2988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:50:40.667665 2988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:50:40.671809 2988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:50:40.672065 2988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:50:40.673053 2988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:50:40.674530 2988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:50:40.674641 2988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:50:40.674748 2988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:50:40.674825 2988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:50:40.705818 2988 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:45185
I20260811 12:50:40.705866 3180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:45185 every 8 connection(s)
I20260811 12:50:40.707326 2988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:50:40.712476 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 2988
Aug 11 12:50:40 dist-test-slave-f0tn krb5kdc[1521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452640, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:50:40.728871 3187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:40.713579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35570 (local address 127.7.111.126:40455)
0811 12:50:40.714380 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:40.714385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:40.714403 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:50:40.714452 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:40.714456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:40.714521 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:40.714634 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:50:40.716264 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.717193 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.718589 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.719152 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.719308 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:40.722697 (+ 3389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:50:40.722702 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:50:40.722705 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:50:40.722739 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:50:40.726233 (+ 3494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:40.727054 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:40.727058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:40.727060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:40.727186 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:40.727583 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:40.727587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:40.727589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:40.727882 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:50:40.728164 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:40.728377 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 12:50:40.728553 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":689,"thread_start_us":556,"threads_started":1}
I20260811 12:50:40.729954 2773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:40.709909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:60829 (local address 127.7.111.126:40455)
0811 12:50:40.710081 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:40.710085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:40.711269 (+ 1184us) server_negotiation.cc:407] Connection header received
0811 12:50:40.713607 (+ 2338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:40.713614 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:40.713705 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:40.713819 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:50:40.715815 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.716794 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.718525 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.719126 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.719245 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:40.723267 (+ 4022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:50:40.723274 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:50:40.723278 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:50:40.723316 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:50:40.726517 (+ 3201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:40.727344 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:40.727347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:40.727349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:40.727457 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:40.727873 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:40.727878 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:40.727881 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:40.728196 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:50:40.728327 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:40.729370 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0811 12:50:40.729607 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":57}
I20260811 12:50:40.730845 3184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:40.710368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.67:60829)
0811 12:50:40.711088 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:40.711133 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:50:40.713261 (+ 2128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:50:40.714295 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:50:40.714307 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:50:40.715014 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:50:40.715645 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:40.715663 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.716967 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.716973 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:40.718388 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:40.718402 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.719360 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.719368 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:40.719681 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:40.720358 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:50:40.720390 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:50:40.723040 (+ 2650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:50:40.726651 (+ 3611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:50:40.726660 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:50:40.726680 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:50:40.727226 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:50:40.727560 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:50:40.727565 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:50:40.727569 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:50:40.727754 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:50:40.728328 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:50:40.728336 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:50:40.728769 (+ 433us) client_negotiation.cc:769] Sending connection context
0811 12:50:40.729054 (+ 285us) client_negotiation.cc:240] Negotiation successful
0811 12:50:40.729641 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":286,"threads_started":1}
I20260811 12:50:40.732161 3182 heartbeater.cc:347] Connected to a master server at 127.7.111.126:40455
I20260811 12:50:40.732445 3182 heartbeater.cc:464] Registering TS with master...
I20260811 12:50:40.733191 3182 heartbeater.cc:511] Master 127.7.111.126:40455 requested a full tablet report, sending...
I20260811 12:50:40.734721 2474 ts_manager.cc:194] Registered new tserver with Master: d99382592a4347488d921cece4145022 (127.7.111.67:45185)
I20260811 12:50:40.735549 2474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:60829
I20260811 12:50:40.743741 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:50:40.750555 2773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:40.744692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35580 (local address 127.7.111.126:40455)
0811 12:50:40.744874 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:40.744880 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:40.745302 (+ 422us) server_negotiation.cc:407] Connection header received
0811 12:50:40.745566 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:40.745570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:40.745714 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:40.745849 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:50:40.746304 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.747044 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.748660 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.749060 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.749171 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:40.749723 (+ 552us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:50:40.749929 (+ 206us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 12:50:40.750306 (+ 377us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35580: token expired
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 12:50:40.750669 2773 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35580: token expired
W20260811 12:50:40.751183 2352 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.7.111.126:40455)
I20260811 12:50:40.763895 2773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:40.751714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35582 (local address 127.7.111.126:40455)
0811 12:50:40.751866 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:40.751870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:40.751886 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:50:40.752001 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:40.752004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:40.752075 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:40.752147 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:50:40.753505 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.754208 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.755608 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.756003 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.756094 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:40.758694 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:50:40.758699 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:50:40.758709 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:50:40.758742 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:50:40.761166 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:40.761919 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:40.761955 (+ 36us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:40.761959 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:40.762140 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:40.762538 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:40.762544 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:40.762547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:40.762813 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:50:40.763058 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:40.763357 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 12:50:40.763589 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 12:50:40.767174 2474 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35582:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:50:40.774463 2474 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:50:40.808054 2770 heartbeater.cc:503] Master 127.7.111.126:40455 was elected leader, sending a full tablet report...
I20260811 12:50:40.808200 3194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:40.785639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:45185 (local address 127.0.0.1:53254)
0811 12:50:40.798584 (+ 12945us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:40.798649 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 12:50:40.798888 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:50:40.799828 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:50:40.799837 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:50:40.799875 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:50:40.800349 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:40.800362 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.802689 (+ 2327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.802698 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:40.805090 (+ 2392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:40.805103 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.807082 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.807088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:40.807352 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:40.807417 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 12:50:40.807535 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 12:50:40.807747 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12637,"mutex_wait_us":1,"thread_start_us":12468,"threads_started":1}
I20260811 12:50:40.809334 3195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:40.785843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53254 (local address 127.7.111.67:45185)
0811 12:50:40.798007 (+ 12164us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:40.798017 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:40.799114 (+ 1097us) server_negotiation.cc:407] Connection header received
0811 12:50:40.799259 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:40.799269 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:40.799572 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:40.799787 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:50:40.800571 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.802485 (+ 1914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.805309 (+ 2824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.806795 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.808469 (+ 1674us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:40.808730 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:40.808818 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 12:50:40.808987 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11974,"thread_start_us":11801,"threads_started":1}
I20260811 12:50:40.811476 3198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:40.786805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58710 (local address 127.7.111.65:44679)
0811 12:50:40.802406 (+ 15601us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:40.802414 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:40.803007 (+ 593us) server_negotiation.cc:407] Connection header received
0811 12:50:40.803184 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:40.803192 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:40.803471 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:40.803689 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:50:40.804627 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.805907 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.808750 (+ 2843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.809946 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.810065 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:40.810832 (+ 767us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:40.810903 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 12:50:40.811062 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15427,"thread_start_us":15267,"threads_started":1}
I20260811 12:50:40.812704 3196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:40.786246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:42023 (local address 127.0.0.1:35950)
0811 12:50:40.803609 (+ 17363us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:40.803627 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:50:40.803742 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:50:40.804655 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:50:40.804659 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:50:40.804683 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:50:40.805085 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:40.805098 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.807010 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.807016 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:40.810115 (+ 3099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:40.810130 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.811934 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.811941 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:40.812100 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:40.812123 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 12:50:40.812210 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 12:50:40.812356 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17196,"thread_start_us":16342,"threads_started":1}
I20260811 12:50:40.812768 3199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:40.787145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35950 (local address 127.7.111.66:42023)
0811 12:50:40.801876 (+ 14731us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:40.801885 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:40.803876 (+ 1991us) server_negotiation.cc:407] Connection header received
0811 12:50:40.803979 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:40.803986 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:40.804207 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:40.804397 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:50:40.805276 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.806864 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.810304 (+ 3440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.811720 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.811845 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:40.812038 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:40.812268 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 12:50:40.812425 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14551,"thread_start_us":14400,"threads_started":1}
I20260811 12:50:40.813416 3069 tablet_service.cc:1467] Processing CreateTablet for tablet a4b6dd312c444a02ba4cc776518781e2 (DEFAULT_TABLE table=test-table [id=cae4fe8a01e24c08a39782a7ab2ee309]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:50:40.814102 3197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:40.785856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:44679 (local address 127.0.0.1:58710)
0811 12:50:40.802873 (+ 17017us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:40.802891 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:50:40.803022 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:50:40.803950 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:50:40.803955 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:50:40.803979 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:50:40.804426 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:40.804439 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.806040 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.806045 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:40.807747 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:40.807761 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:40.810170 (+ 2409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:40.810176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:40.810333 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:40.810364 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:50:40.810444 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 12:50:40.813741 (+ 3297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16892,"mutex_wait_us":24,"thread_start_us":17417,"threads_started":1}
I20260811 12:50:40.814895 3069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4b6dd312c444a02ba4cc776518781e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:50:40.817047 2878 tablet_service.cc:1467] Processing CreateTablet for tablet a4b6dd312c444a02ba4cc776518781e2 (DEFAULT_TABLE table=test-table [id=cae4fe8a01e24c08a39782a7ab2ee309]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:50:40.817625 2666 tablet_service.cc:1467] Processing CreateTablet for tablet a4b6dd312c444a02ba4cc776518781e2 (DEFAULT_TABLE table=test-table [id=cae4fe8a01e24c08a39782a7ab2ee309]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:50:40.818487 2878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4b6dd312c444a02ba4cc776518781e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:50:40.818997 2666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4b6dd312c444a02ba4cc776518781e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:50:40.827844 3201 tablet_bootstrap.cc:492] T a4b6dd312c444a02ba4cc776518781e2 P d99382592a4347488d921cece4145022: Bootstrap starting.
I20260811 12:50:40.830978 3201 tablet_bootstrap.cc:654] T a4b6dd312c444a02ba4cc776518781e2 P d99382592a4347488d921cece4145022: Neither blocks nor log segments found. Creating new log.
I20260811 12:50:40.832000 3201 log.cc:824] T a4b6dd312c444a02ba4cc776518781e2 P d99382592a4347488d921cece4145022: Log is configured to *not* fsync() on all Append() calls
I20260811 12:50:40.834820 3203 tablet_bootstrap.cc:492] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb: Bootstrap starting.
I20260811 12:50:40.836472 3201 tablet_bootstrap.cc:492] T a4b6dd312c444a02ba4cc776518781e2 P d99382592a4347488d921cece4145022: No bootstrap required, opened a new log
I20260811 12:50:40.836761 3201 ts_tablet_manager.cc:1397] T a4b6dd312c444a02ba4cc776518781e2 P d99382592a4347488d921cece4145022: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 12:50:40.838541 3203 tablet_bootstrap.cc:654] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb: Neither blocks nor log segments found. Creating new log.
I20260811 12:50:40.834117 3205 tablet_bootstrap.cc:492] T a4b6dd312c444a02ba4cc776518781e2 P 66dcdc14b4cc4517b3b044a53d900d0d: Bootstrap starting.
I20260811 12:50:40.839821 3203 log.cc:824] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb: Log is configured to *not* fsync() on all Append() calls
I20260811 12:50:40.841884 3205 tablet_bootstrap.cc:654] T a4b6dd312c444a02ba4cc776518781e2 P 66dcdc14b4cc4517b3b044a53d900d0d: Neither blocks nor log segments found. Creating new log.
I20260811 12:50:40.843145 3203 tablet_bootstrap.cc:492] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb: No bootstrap required, opened a new log
I20260811 12:50:40.843281 3205 log.cc:824] T a4b6dd312c444a02ba4cc776518781e2 P 66dcdc14b4cc4517b3b044a53d900d0d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:50:40.843441 3203 ts_tablet_manager.cc:1397] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 12:50:40.845985 3205 tablet_bootstrap.cc:492] T a4b6dd312c444a02ba4cc776518781e2 P 66dcdc14b4cc4517b3b044a53d900d0d: No bootstrap required, opened a new log
I20260811 12:50:40.846251 3205 ts_tablet_manager.cc:1397] T a4b6dd312c444a02ba4cc776518781e2 P 66dcdc14b4cc4517b3b044a53d900d0d: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.005s
I20260811 12:50:40.846895 3201 raft_consensus.cc:348] T a4b6dd312c444a02ba4cc776518781e2 P d99382592a4347488d921cece4145022 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99382592a4347488d921cece4145022" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45185 } } peers { permanent_uuid: "c2766f606ebd4d81a30334721da429eb" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 44679 } } peers { permanent_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42023 } }
I20260811 12:50:40.847386 3201 raft_consensus.cc:374] T a4b6dd312c444a02ba4cc776518781e2 P d99382592a4347488d921cece4145022 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:50:40.847539 3201 raft_consensus.cc:729] T a4b6dd312c444a02ba4cc776518781e2 P d99382592a4347488d921cece4145022 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d99382592a4347488d921cece4145022, State: Initialized, Role: FOLLOWER
I20260811 12:50:40.848225 3201 consensus_queue.cc:260] T a4b6dd312c444a02ba4cc776518781e2 P d99382592a4347488d921cece4145022 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99382592a4347488d921cece4145022" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45185 } } peers { permanent_uuid: "c2766f606ebd4d81a30334721da429eb" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 44679 } } peers { permanent_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42023 } }
I20260811 12:50:40.849421 3182 heartbeater.cc:503] Master 127.7.111.126:40455 was elected leader, sending a full tablet report...
I20260811 12:50:40.851468 3201 ts_tablet_manager.cc:1428] T a4b6dd312c444a02ba4cc776518781e2 P d99382592a4347488d921cece4145022: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260811 12:50:40.853353 3203 raft_consensus.cc:348] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99382592a4347488d921cece4145022" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45185 } } peers { permanent_uuid: "c2766f606ebd4d81a30334721da429eb" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 44679 } } peers { permanent_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42023 } }
I20260811 12:50:40.853945 3203 raft_consensus.cc:374] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:50:40.854112 3203 raft_consensus.cc:729] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2766f606ebd4d81a30334721da429eb, State: Initialized, Role: FOLLOWER
I20260811 12:50:40.854756 3203 consensus_queue.cc:260] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99382592a4347488d921cece4145022" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45185 } } peers { permanent_uuid: "c2766f606ebd4d81a30334721da429eb" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 44679 } } peers { permanent_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42023 } }
I20260811 12:50:40.854866 3205 raft_consensus.cc:348] T a4b6dd312c444a02ba4cc776518781e2 P 66dcdc14b4cc4517b3b044a53d900d0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99382592a4347488d921cece4145022" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45185 } } peers { permanent_uuid: "c2766f606ebd4d81a30334721da429eb" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 44679 } } peers { permanent_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42023 } }
I20260811 12:50:40.855305 3205 raft_consensus.cc:374] T a4b6dd312c444a02ba4cc776518781e2 P 66dcdc14b4cc4517b3b044a53d900d0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:50:40.855424 3205 raft_consensus.cc:729] T a4b6dd312c444a02ba4cc776518781e2 P 66dcdc14b4cc4517b3b044a53d900d0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66dcdc14b4cc4517b3b044a53d900d0d, State: Initialized, Role: FOLLOWER
I20260811 12:50:40.856154 3205 consensus_queue.cc:260] T a4b6dd312c444a02ba4cc776518781e2 P 66dcdc14b4cc4517b3b044a53d900d0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99382592a4347488d921cece4145022" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45185 } } peers { permanent_uuid: "c2766f606ebd4d81a30334721da429eb" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 44679 } } peers { permanent_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42023 } }
I20260811 12:50:40.857445 2977 heartbeater.cc:503] Master 127.7.111.126:40455 was elected leader, sending a full tablet report...
I20260811 12:50:40.857985 3205 ts_tablet_manager.cc:1428] T a4b6dd312c444a02ba4cc776518781e2 P 66dcdc14b4cc4517b3b044a53d900d0d: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260811 12:50:40.859287 3203 ts_tablet_manager.cc:1428] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb: Time spent starting tablet: real 0.016s user 0.005s sys 0.011s
W20260811 12:50:40.958911 3183 tablet.cc:2369] T a4b6dd312c444a02ba4cc776518781e2 P d99382592a4347488d921cece4145022: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:50:40.991544 2978 tablet.cc:2369] T a4b6dd312c444a02ba4cc776518781e2 P 66dcdc14b4cc4517b3b044a53d900d0d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:50:41.024336 2772 tablet.cc:2369] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:50:41.164803 3212 raft_consensus.cc:482] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:50:41.165144 3212 raft_consensus.cc:504] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99382592a4347488d921cece4145022" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45185 } } peers { permanent_uuid: "c2766f606ebd4d81a30334721da429eb" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 44679 } } peers { permanent_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42023 } }
I20260811 12:50:41.168205 3212 leader_election.cc:288] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d99382592a4347488d921cece4145022 (127.7.111.67:45185), 66dcdc14b4cc4517b3b044a53d900d0d (127.7.111.66:42023)
I20260811 12:50:41.175644 3199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:41.167580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:59489 (local address 127.7.111.66:42023)
0811 12:50:41.167797 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:41.167803 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:41.168512 (+ 709us) server_negotiation.cc:407] Connection header received
0811 12:50:41.168786 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:41.168792 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:41.168877 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:41.168962 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:50:41.169644 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:41.170695 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:41.173371 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:41.174593 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:41.174757 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:41.174833 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:41.175253 (+ 420us) server_negotiation.cc:299] Negotiation successful
0811 12:50:41.175384 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:50:41.175645 3195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:41.167643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:59911 (local address 127.7.111.67:45185)
0811 12:50:41.167876 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:41.167882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:41.168893 (+ 1011us) server_negotiation.cc:407] Connection header received
0811 12:50:41.169085 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:41.169091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:41.169168 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:41.169260 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:50:41.169860 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:41.170859 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:41.173371 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:41.174592 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:41.174757 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:41.174833 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:41.175254 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 12:50:41.175384 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 12:50:41.175904 3226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:41.167366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:45185 (local address 127.7.111.65:59911)
0811 12:50:41.168782 (+ 1416us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:41.168800 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:50:41.168926 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:50:41.169281 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:50:41.169285 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:50:41.169305 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:50:41.169700 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:41.169710 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:41.170985 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:41.170994 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:41.173202 (+ 2208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:41.173211 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:41.174844 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:41.174849 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:41.175074 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:41.175123 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 12:50:41.175220 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 12:50:41.175428 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1297,"mutex_wait_us":681,"thread_start_us":532,"threads_started":1}
I20260811 12:50:41.175937 3222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:41.167366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:42023 (local address 127.7.111.65:59489)
0811 12:50:41.168404 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0811 12:50:41.168428 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:50:41.168624 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:50:41.168984 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:50:41.168988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:50:41.169015 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:50:41.169506 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:41.169522 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:41.170828 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:41.170832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:41.173196 (+ 2364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:50:41.173211 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:50:41.174819 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:50:41.174830 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:50:41.175070 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:41.175127 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 12:50:41.175221 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 12:50:41.175428 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":893,"mutex_wait_us":79,"thread_start_us":617,"threads_started":1}
I20260811 12:50:41.176999 3102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b6dd312c444a02ba4cc776518781e2" candidate_uuid: "c2766f606ebd4d81a30334721da429eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d99382592a4347488d921cece4145022" is_pre_election: true
I20260811 12:50:41.176996 2910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b6dd312c444a02ba4cc776518781e2" candidate_uuid: "c2766f606ebd4d81a30334721da429eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" is_pre_election: true
I20260811 12:50:41.177493 3102 raft_consensus.cc:2454] T a4b6dd312c444a02ba4cc776518781e2 P d99382592a4347488d921cece4145022 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2766f606ebd4d81a30334721da429eb in term 0.
I20260811 12:50:41.177491 2910 raft_consensus.cc:2454] T a4b6dd312c444a02ba4cc776518781e2 P 66dcdc14b4cc4517b3b044a53d900d0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2766f606ebd4d81a30334721da429eb in term 0.
I20260811 12:50:41.178097 2598 leader_election.cc:302] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66dcdc14b4cc4517b3b044a53d900d0d, c2766f606ebd4d81a30334721da429eb; no voters:
I20260811 12:50:41.178481 3212 raft_consensus.cc:2781] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:50:41.178601 3212 raft_consensus.cc:482] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:50:41.178695 3212 raft_consensus.cc:3037] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:50:41.181530 3212 raft_consensus.cc:504] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99382592a4347488d921cece4145022" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45185 } } peers { permanent_uuid: "c2766f606ebd4d81a30334721da429eb" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 44679 } } peers { permanent_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42023 } }
I20260811 12:50:41.182227 3212 leader_election.cc:288] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb [CANDIDATE]: Term 1 election: Requested vote from peers d99382592a4347488d921cece4145022 (127.7.111.67:45185), 66dcdc14b4cc4517b3b044a53d900d0d (127.7.111.66:42023)
I20260811 12:50:41.182658 3102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b6dd312c444a02ba4cc776518781e2" candidate_uuid: "c2766f606ebd4d81a30334721da429eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d99382592a4347488d921cece4145022"
I20260811 12:50:41.182718 2910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b6dd312c444a02ba4cc776518781e2" candidate_uuid: "c2766f606ebd4d81a30334721da429eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66dcdc14b4cc4517b3b044a53d900d0d"
I20260811 12:50:41.182904 2910 raft_consensus.cc:3037] T a4b6dd312c444a02ba4cc776518781e2 P 66dcdc14b4cc4517b3b044a53d900d0d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:50:41.182893 3102 raft_consensus.cc:3037] T a4b6dd312c444a02ba4cc776518781e2 P d99382592a4347488d921cece4145022 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:50:41.185272 2910 raft_consensus.cc:2454] T a4b6dd312c444a02ba4cc776518781e2 P 66dcdc14b4cc4517b3b044a53d900d0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2766f606ebd4d81a30334721da429eb in term 1.
I20260811 12:50:41.185276 3102 raft_consensus.cc:2454] T a4b6dd312c444a02ba4cc776518781e2 P d99382592a4347488d921cece4145022 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2766f606ebd4d81a30334721da429eb in term 1.
I20260811 12:50:41.185719 2600 leader_election.cc:302] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 66dcdc14b4cc4517b3b044a53d900d0d, c2766f606ebd4d81a30334721da429eb, d99382592a4347488d921cece4145022; no voters:
I20260811 12:50:41.186018 3212 raft_consensus.cc:2781] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:50:41.186470 3212 raft_consensus.cc:686] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb [term 1 LEADER]: Becoming Leader. State: Replica: c2766f606ebd4d81a30334721da429eb, State: Running, Role: LEADER
I20260811 12:50:41.186926 3212 consensus_queue.cc:237] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99382592a4347488d921cece4145022" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45185 } } peers { permanent_uuid: "c2766f606ebd4d81a30334721da429eb" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 44679 } } peers { permanent_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42023 } }
I20260811 12:50:41.191397 2473 catalog_manager.cc:5495] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb reported cstate change: term changed from 0 to 1, leader changed from <none> to c2766f606ebd4d81a30334721da429eb (127.7.111.65). New cstate: current_term: 1 leader_uuid: "c2766f606ebd4d81a30334721da429eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99382592a4347488d921cece4145022" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2766f606ebd4d81a30334721da429eb" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42023 } health_report { overall_health: UNKNOWN } } }
I20260811 12:50:41.270231 3198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:41.263919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58712 (local address 127.7.111.65:44679)
0811 12:50:41.264143 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:41.264150 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:41.264173 (+ 23us) server_negotiation.cc:407] Connection header received
0811 12:50:41.264285 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:41.264290 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:41.264406 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:41.264548 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:50:41.264880 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:41.266039 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:41.267504 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:41.268207 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:41.268331 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:41.268838 (+ 507us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:50:41.269542 (+ 704us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:50:41.269681 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:41.269864 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 12:50:41.269954 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:50:41.278695 3102 raft_consensus.cc:1261] T a4b6dd312c444a02ba4cc776518781e2 P d99382592a4347488d921cece4145022 [term 1 FOLLOWER]: Refusing update from remote peer c2766f606ebd4d81a30334721da429eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:50:41.278707 2910 raft_consensus.cc:1261] T a4b6dd312c444a02ba4cc776518781e2 P 66dcdc14b4cc4517b3b044a53d900d0d [term 1 FOLLOWER]: Refusing update from remote peer c2766f606ebd4d81a30334721da429eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:50:41.279644 3212 consensus_queue.cc:1035] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "66dcdc14b4cc4517b3b044a53d900d0d" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 42023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:50:41.279949 3229 consensus_queue.cc:1035] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d99382592a4347488d921cece4145022" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 45185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:50:41.290542 3233 mvcc.cc:205] Tried to move back new op lower bound from 7317310018667991040 to 7317310018305118208. Current Snapshot: MvccSnapshot[applied={T|T < 7317310018667991040}]
I20260811 12:50:41.303185 2474 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:50:41.303442 2474 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:50:41.306937 2474 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb: Sending DeleteTablet for 3 replicas of tablet a4b6dd312c444a02ba4cc776518781e2
I20260811 12:50:41.308161 3069 tablet_service.cc:1514] Processing DeleteTablet for tablet a4b6dd312c444a02ba4cc776518781e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:50:41 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:53254
I20260811 12:50:41.308223 2878 tablet_service.cc:1514] Processing DeleteTablet for tablet a4b6dd312c444a02ba4cc776518781e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:50:41 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:35950
I20260811 12:50:41.308413 2666 tablet_service.cc:1514] Processing DeleteTablet for tablet a4b6dd312c444a02ba4cc776518781e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:50:41 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:58710
I20260811 12:50:41.309927 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 2542
I20260811 12:50:41.320555 3253 tablet_replica.cc:330] T a4b6dd312c444a02ba4cc776518781e2 P d99382592a4347488d921cece4145022: stopping tablet replica
I20260811 12:50:41.321255 3253 raft_consensus.cc:2229] T a4b6dd312c444a02ba4cc776518781e2 P d99382592a4347488d921cece4145022 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:50:41.321981 3253 raft_consensus.cc:2258] T a4b6dd312c444a02ba4cc776518781e2 P d99382592a4347488d921cece4145022 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:50:41.324599 3253 ts_tablet_manager.cc:1905] T a4b6dd312c444a02ba4cc776518781e2 P d99382592a4347488d921cece4145022: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:50:41.331947 3253 ts_tablet_manager.cc:1918] T a4b6dd312c444a02ba4cc776518781e2 P d99382592a4347488d921cece4145022: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:50:41.332228 3253 log.cc:1196] T a4b6dd312c444a02ba4cc776518781e2 P d99382592a4347488d921cece4145022: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/a4b6dd312c444a02ba4cc776518781e2
I20260811 12:50:41.332921 3253 ts_tablet_manager.cc:1939] T a4b6dd312c444a02ba4cc776518781e2 P d99382592a4347488d921cece4145022: Deleting consensus metadata
I20260811 12:50:41.335175 2458 catalog_manager.cc:4841] TS d99382592a4347488d921cece4145022 (127.7.111.67:45185): tablet a4b6dd312c444a02ba4cc776518781e2 (table test-table [id=cae4fe8a01e24c08a39782a7ab2ee309]) successfully deleted
I20260811 12:50:41.337984 3254 tablet_replica.cc:330] T a4b6dd312c444a02ba4cc776518781e2 P 66dcdc14b4cc4517b3b044a53d900d0d: stopping tablet replica
I20260811 12:50:41.338599 3254 raft_consensus.cc:2229] T a4b6dd312c444a02ba4cc776518781e2 P 66dcdc14b4cc4517b3b044a53d900d0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:50:41.339344 3254 raft_consensus.cc:2258] T a4b6dd312c444a02ba4cc776518781e2 P 66dcdc14b4cc4517b3b044a53d900d0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:50:41.340265 3255 tablet_replica.cc:330] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb: stopping tablet replica
I20260811 12:50:41.340762 3255 raft_consensus.cc:2229] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:50:41.341611 3255 raft_consensus.cc:2258] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:50:41.342120 3254 ts_tablet_manager.cc:1905] T a4b6dd312c444a02ba4cc776518781e2 P 66dcdc14b4cc4517b3b044a53d900d0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:50:41.344527 3255 ts_tablet_manager.cc:1905] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:50:41.348222 3254 ts_tablet_manager.cc:1918] T a4b6dd312c444a02ba4cc776518781e2 P 66dcdc14b4cc4517b3b044a53d900d0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:50:41.348415 3254 log.cc:1196] T a4b6dd312c444a02ba4cc776518781e2 P 66dcdc14b4cc4517b3b044a53d900d0d: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/a4b6dd312c444a02ba4cc776518781e2
I20260811 12:50:41.349036 3254 ts_tablet_manager.cc:1939] T a4b6dd312c444a02ba4cc776518781e2 P 66dcdc14b4cc4517b3b044a53d900d0d: Deleting consensus metadata
I20260811 12:50:41.351950 2457 catalog_manager.cc:4841] TS 66dcdc14b4cc4517b3b044a53d900d0d (127.7.111.66:42023): tablet a4b6dd312c444a02ba4cc776518781e2 (table test-table [id=cae4fe8a01e24c08a39782a7ab2ee309]) successfully deleted
I20260811 12:50:41.352192 3255 ts_tablet_manager.cc:1918] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:50:41.352393 3255 log.cc:1196] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/a4b6dd312c444a02ba4cc776518781e2
I20260811 12:50:41.352950 3255 ts_tablet_manager.cc:1939] T a4b6dd312c444a02ba4cc776518781e2 P c2766f606ebd4d81a30334721da429eb: Deleting consensus metadata
I20260811 12:50:41.354792 2456 catalog_manager.cc:4841] TS c2766f606ebd4d81a30334721da429eb (127.7.111.65:44679): tablet a4b6dd312c444a02ba4cc776518781e2 (table test-table [id=cae4fe8a01e24c08a39782a7ab2ee309]) successfully deleted
I20260811 12:50:41.354972 3198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:41.311159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58728 (local address 127.7.111.65:44679)
0811 12:50:41.311368 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:41.311374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:41.334724 (+ 23350us) server_negotiation.cc:407] Connection header received
0811 12:50:41.335054 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:41.335060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:41.335161 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:41.335267 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:50:41.337925 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:41.339339 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:41.341316 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:41.342034 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:41.342139 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:41.347432 (+ 5293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:50:41.347447 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:50:41.347466 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:50:41.347518 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:50:41.351589 (+ 4071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:41.352640 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:41.352648 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:41.352650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:41.352809 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:41.353249 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:41.353253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:41.353255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:41.353660 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:50:41.353982 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:41.354331 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 12:50:41.354627 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 12:50:41.355708 2762 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:50:41.455988 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 2542
I20260811 12:50:41.484818 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 2780
I20260811 12:50:41.497980 3199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:41.485568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35964 (local address 127.7.111.66:42023)
0811 12:50:41.485741 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:41.485747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:41.485816 (+ 69us) server_negotiation.cc:407] Connection header received
0811 12:50:41.486019 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:41.486024 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:41.486105 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:41.486190 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:50:41.487586 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:41.488314 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:41.489712 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:41.490127 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:41.490249 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:41.492764 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:50:41.492777 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:50:41.492793 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:50:41.492825 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:50:41.495284 (+ 2459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:41.495968 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:41.495975 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:41.495977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:41.496107 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:41.496472 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:41.496476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:41.496478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:41.496846 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:50:41.497127 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:41.497453 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 12:50:41.497667 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 12:50:41.498593 2965 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:50:41.585983 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 2780
I20260811 12:50:41.611040 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 2988
I20260811 12:50:41.624601 3195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:50:41.611821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53266 (local address 127.7.111.67:45185)
0811 12:50:41.612068 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 12:50:41.612074 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:50:41.612097 (+ 23us) server_negotiation.cc:407] Connection header received
0811 12:50:41.612252 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:50:41.612256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:50:41.612339 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:50:41.612434 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:50:41.613892 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:41.614631 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:41.615927 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:50:41.616388 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:50:41.616499 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:50:41.619231 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:50:41.619248 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:50:41.619268 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:50:41.619301 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:50:41.621886 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:41.622564 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:41.622572 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:41.622574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:41.622717 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:50:41.623066 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:50:41.623070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:50:41.623072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:50:41.623448 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:50:41.623728 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 12:50:41.624040 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 12:50:41.624273 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 12:50:41.625265 3173 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:50:41.712420 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 2988
I20260811 12:50:41.737142 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 2424
I20260811 12:50:41.738329 2531 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:50:41.851854 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 2424
I20260811 12:51:01.879701 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:40455
--webserver_interface=127.7.111.126
--webserver_port=45933
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:44687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:40455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:51:02.222507 3269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:02.222824 3269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:02.222965 3269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:02.230302 3269 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 12:51:02.230419 3269 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 12:51:02.230484 3269 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 12:51:02.230553 3269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:51:02.230633 3269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:02.230728 3269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:02.230792 3269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:51:02.230868 3269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:51:02.241111 3269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:44687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:40455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:40455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=45933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:02.242507 3269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:02.243888 3269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:02.256059 3274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:02.255993 3275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:02.255968 3277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:02.256721 3269 server_base.cc:1034] running on GCE node
I20260811 12:51:02.257512 3269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:02.258904 3269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:02.260107 3269 hybrid_clock.cc:648] HybridClock initialized: now 1786452662260054 us; error 33 us; skew 500 ppm
I20260811 12:51:02.260448 3269 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:51:02 dist-test-slave-f0tn krb5kdc[1521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452662, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:51:02.267846 3269 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:51:02.269502 3269 webserver.cc:460] Webserver started at http://127.7.111.126:45933/ using document root <none> and password file <none>
I20260811 12:51:02.270249 3269 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:02.270345 3269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:02.275941 3269 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:51:02.278865 3286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:02.280210 3269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:51:02.280397 3269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "17ccc579f9934760a2dcd371e1c026fb"
format_stamp: "Formatted at 2026-08-11 12:50:15 on dist-test-slave-f0tn"
server_key: "7183e2e070a64cf53f27eb77929542e5"
server_key_iv: "4aad647f02b61c7b03459e2a3b25302e"
server_key_version: "encryptionkey@0"
I20260811 12:51:02.281369 3269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:02.310741 3269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:02.315102 3269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:02.315349 3269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:02.333642 3269 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:40455
I20260811 12:51:02.333662 3365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:40455 every 8 connection(s)
I20260811 12:51:02.335213 3269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:51:02.341866 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 3269
I20260811 12:51:02.342720 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:44679
--local_ip_for_outbound_sockets=127.7.111.65
--tserver_master_addrs=127.7.111.126:40455
--webserver_port=34863
--webserver_interface=127.7.111.65
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:44687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:51:02.343001 3366 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:51:02.346508 3366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb: Bootstrap starting.
I20260811 12:51:02.368275 3366 log.cc:824] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:02.386909 3366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:51:02.387549 3366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb: Bootstrap complete.
I20260811 12:51:02.397446 3366 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ccc579f9934760a2dcd371e1c026fb" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40455 } }
I20260811 12:51:02.398432 3366 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17ccc579f9934760a2dcd371e1c026fb, State: Initialized, Role: FOLLOWER
I20260811 12:51:02.399111 3366 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ccc579f9934760a2dcd371e1c026fb" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40455 } }
I20260811 12:51:02.399369 3366 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:51:02.399504 3366 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:51:02.399685 3366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 2 FOLLOWER]: Advancing to term 3
I20260811 12:51:02.403723 3366 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ccc579f9934760a2dcd371e1c026fb" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40455 } }
I20260811 12:51:02.404223 3366 leader_election.cc:302] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17ccc579f9934760a2dcd371e1c026fb; no voters:
I20260811 12:51:02.404927 3366 leader_election.cc:288] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 12:51:02.405097 3375 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 3 FOLLOWER]: Leader election won for term 3
I20260811 12:51:02.406234 3375 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [term 3 LEADER]: Becoming Leader. State: Replica: 17ccc579f9934760a2dcd371e1c026fb, State: Running, Role: LEADER
I20260811 12:51:02.406879 3375 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ccc579f9934760a2dcd371e1c026fb" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40455 } }
I20260811 12:51:02.407266 3366 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:51:02.409523 3379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17ccc579f9934760a2dcd371e1c026fb. Latest consensus state: current_term: 3 leader_uuid: "17ccc579f9934760a2dcd371e1c026fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ccc579f9934760a2dcd371e1c026fb" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40455 } } }
I20260811 12:51:02.409940 3379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:02.410418 3377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "17ccc579f9934760a2dcd371e1c026fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ccc579f9934760a2dcd371e1c026fb" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40455 } } }
I20260811 12:51:02.410758 3377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:02.413178 3389 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:51:02.422362 3389 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:51:02.423946 3389 catalog_manager.cc:1259] Loaded cluster ID: f96b0f04889143ec87815c16db24b363
I20260811 12:51:02.424135 3389 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:51:02.428419 3389 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:51:02.430183 3389 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb: Loaded TSK: 0
I20260811 12:51:02.430439 3389 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 17ccc579f9934760a2dcd371e1c026fb: Loaded TSK: 1
F20260811 12:51:02.431648 3389 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0ebbf679ad google::LogMessage::Fail() at ??:0
@ 0x7f0ebbf6a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f0ebbf674de google::LogMessage::Flush() at ??:0
@ 0x7f0ebbf68a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0ebfccb7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f0ebfcba73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f0ed199a1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f0ed1999783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f0ed1a252a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f0ed1a250c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f0ed1a6f5e9 std::function<>::operator()() at ??:0
@ 0x7f0ed199dcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f0ed199bc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0ed1a23610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f0ed1a23439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f0ebebb3345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0ebebb691c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f0ebebb6799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f0ebeb91eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0ec1e6b6db start_thread at ??:0
@ 0x7f0ebae0161f clone at ??:0
W20260811 12:51:02.730511 3373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:02.730806 3373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:02.730937 3373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:02.738193 3373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:02.738317 3373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:02.738476 3373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:51:02.748364 3373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:44687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:44679
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=34863
--tserver_master_addrs=127.7.111.126:40455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:02.749666 3373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:02.750962 3373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:02.763446 3453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:02.763541 3450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:02.763501 3451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:02.764284 3373 server_base.cc:1034] running on GCE node
I20260811 12:51:02.764848 3373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:02.765769 3373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:02.766970 3373 hybrid_clock.cc:648] HybridClock initialized: now 1786452662766921 us; error 43 us; skew 500 ppm
I20260811 12:51:02.767275 3373 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:51:02 dist-test-slave-f0tn krb5kdc[1521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452662, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:51:02.770428 3373 init.cc:377] Logged in from keytab as kudu/127.7.111.65@KRBTEST.COM (short username kudu)
I20260811 12:51:02.771983 3373 webserver.cc:460] Webserver started at http://127.7.111.65:34863/ using document root <none> and password file <none>
I20260811 12:51:02.772655 3373 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:02.772761 3373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:02.777987 3373 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:51:02.780551 3462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:02.781878 3373 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:51:02.782034 3373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "c2766f606ebd4d81a30334721da429eb"
format_stamp: "Formatted at 2026-08-11 12:50:16 on dist-test-slave-f0tn"
server_key: "d98424a089994169da8f3065415057f6"
server_key_iv: "9a4105b73c3f2ef0ec114fab74f642ea"
server_key_version: "encryptionkey@0"
I20260811 12:51:02.782970 3373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:02.802951 3373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:02.806720 3373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:02.806962 3373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:02.807950 3373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:51:02.809305 3373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:51:02.809422 3373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:02.809543 3373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:51:02.809620 3373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:02.841573 3373 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:44679
I20260811 12:51:02.841684 3651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:44679 every 8 connection(s)
I20260811 12:51:02.843094 3373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:51:02.845911 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 3373
I20260811 12:51:02.846678 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:42023
--local_ip_for_outbound_sockets=127.7.111.66
--tserver_master_addrs=127.7.111.126:40455
--webserver_port=34271
--webserver_interface=127.7.111.66
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:44687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:51:02.848376 3655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:02.846002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.65:40113)
0811 12:51:02.846919 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:02.847191 (+ 272us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":309,"threads_started":1}
W20260811 12:51:02.849135 3653 heartbeater.cc:644] Failed to heartbeat to 127.7.111.126:40455 (0 consecutive failures): Network error: Failed to ping master at 127.7.111.126:40455: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
I20260811 12:51:02.850049 3655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:02.849677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.65:33033)
0811 12:51:02.849826 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:02.849914 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 12:51:02.850919 3655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:02.850568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.65:56339)
0811 12:51:02.850698 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:02.850791 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 12:51:02.851150 3653 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 12:51:03.185307 3658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:03.185623 3658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:03.185731 3658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:03.192952 3658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:03.193070 3658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:03.193182 3658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:51:03.202945 3658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:44687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:42023
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=34271
--tserver_master_addrs=127.7.111.126:40455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:03.204212 3658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:03.205524 3658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:03.218461 3673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:03.218550 3671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:03.218513 3668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:03.219547 3658 server_base.cc:1034] running on GCE node
I20260811 12:51:03.220170 3658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:03.221089 3658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:03.222321 3658 hybrid_clock.cc:648] HybridClock initialized: now 1786452663222247 us; error 50 us; skew 500 ppm
I20260811 12:51:03.222659 3658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:51:03 dist-test-slave-f0tn krb5kdc[1521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452663, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:51:03.225870 3658 init.cc:377] Logged in from keytab as kudu/127.7.111.66@KRBTEST.COM (short username kudu)
I20260811 12:51:03.227423 3658 webserver.cc:460] Webserver started at http://127.7.111.66:34271/ using document root <none> and password file <none>
I20260811 12:51:03.228142 3658 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:03.228266 3658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:03.233811 3658 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:51:03.236547 3684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:03.238047 3658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:51:03.238211 3658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "66dcdc14b4cc4517b3b044a53d900d0d"
format_stamp: "Formatted at 2026-08-11 12:50:17 on dist-test-slave-f0tn"
server_key: "033a81ef562d2557a07e95f58152f6f5"
server_key_iv: "cccf2fa4f344d0e2f7f390517e0c37f0"
server_key_version: "encryptionkey@0"
I20260811 12:51:03.239203 3658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:03.269181 3658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:03.273695 3658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:03.273972 3658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:03.275007 3658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:51:03.276410 3658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:51:03.276510 3658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:03.276621 3658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:51:03.276702 3658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:03.307487 3658 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:42023
I20260811 12:51:03.307547 3865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:42023 every 8 connection(s)
I20260811 12:51:03.308972 3658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:51:03.313850 3872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:03.311923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.66:47617)
0811 12:51:03.312685 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:03.312928 (+ 243us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":291,"threads_started":1}
W20260811 12:51:03.314579 3866 heartbeater.cc:644] Failed to heartbeat to 127.7.111.126:40455 (0 consecutive failures): Network error: Failed to ping master at 127.7.111.126:40455: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
I20260811 12:51:03.315383 3872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:03.315071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.66:41089)
0811 12:51:03.315208 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:03.315279 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 12:51:03.316179 3872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:03.315872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.66:39951)
0811 12:51:03.315995 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:03.316073 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 12:51:03.316411 3866 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 12:51:03.318307 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 3658
I20260811 12:51:03.318977 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:45185
--local_ip_for_outbound_sockets=127.7.111.67
--tserver_master_addrs=127.7.111.126:40455
--webserver_port=33979
--webserver_interface=127.7.111.67
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:44687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:51:03.665161 3874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:03.665467 3874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:03.665617 3874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:03.673262 3874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:03.673372 3874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:03.673555 3874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:51:03.683395 3874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:44687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:45185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=33979
--tserver_master_addrs=127.7.111.126:40455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:03.684626 3874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:03.685961 3874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:03.698783 3880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:03.698859 3884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:03.698819 3881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:03.700066 3874 server_base.cc:1034] running on GCE node
I20260811 12:51:03.700659 3874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:03.701622 3874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:03.702818 3874 hybrid_clock.cc:648] HybridClock initialized: now 1786452663702773 us; error 45 us; skew 500 ppm
I20260811 12:51:03.703111 3874 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:51:03 dist-test-slave-f0tn krb5kdc[1521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452663, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:51:03.706154 3874 init.cc:377] Logged in from keytab as kudu/127.7.111.67@KRBTEST.COM (short username kudu)
I20260811 12:51:03.707499 3874 webserver.cc:460] Webserver started at http://127.7.111.67:33979/ using document root <none> and password file <none>
I20260811 12:51:03.708151 3874 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:03.708271 3874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:03.713497 3874 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:51:03.716091 3896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:03.717334 3874 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:51:03.717501 3874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "d99382592a4347488d921cece4145022"
format_stamp: "Formatted at 2026-08-11 12:50:17 on dist-test-slave-f0tn"
server_key: "c4d5aacbceee4de22be82aaed2c87f01"
server_key_iv: "a3e8a8e6e6ecc2a199b9724cd9d36c45"
server_key_version: "encryptionkey@0"
I20260811 12:51:03.718441 3874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:03.748359 3874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:03.752714 3874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:03.752944 3874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:03.753922 3874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:51:03.755187 3874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:51:03.755273 3874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:03.755352 3874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:51:03.755404 3874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:03.785341 3874 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:45185
I20260811 12:51:03.785423 4083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:45185 every 8 connection(s)
I20260811 12:51:03.786849 3874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:51:03.790994 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 3874
I20260811 12:51:03.791636 4088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:03.789778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.67:54287)
0811 12:51:03.790486 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:03.790740 (+ 254us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":263,"threads_started":1}
W20260811 12:51:03.792445 4086 heartbeater.cc:644] Failed to heartbeat to 127.7.111.126:40455 (0 consecutive failures): Network error: Failed to ping master at 127.7.111.126:40455: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
I20260811 12:51:03.793434 4088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:03.793029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.67:42639)
0811 12:51:03.793186 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:03.793268 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 12:51:03.794250 4088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:03.793906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.67:54113)
0811 12:51:03.794047 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:03.794126 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 12:51:03.794456 4086 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 12:51:03.852344 4090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:03.851592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.65:33907)
0811 12:51:03.852022 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:03.852126 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":204,"threads_started":1}
I20260811 12:51:04.317800 4093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:04.316972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.66:59781)
0811 12:51:04.317458 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:04.317551 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":211,"threads_started":1}
I20260811 12:51:04.795666 4097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:04.794900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.67:41199)
0811 12:51:04.795358 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:04.795444 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":216,"threads_started":1}
I20260811 12:51:04.853816 4099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:04.853119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.65:42663)
0811 12:51:04.853576 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:04.853655 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":206,"threads_started":1}
I20260811 12:51:05.319377 4101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:05.318651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.66:57053)
0811 12:51:05.319121 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:05.319207 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":239,"threads_started":1}
I20260811 12:51:05.797122 4102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:05.796439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.67:38797)
0811 12:51:05.796869 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:05.796947 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":226,"threads_started":1}
I20260811 12:51:05.855497 4103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:05.854751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.65:35893)
0811 12:51:05.855228 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:05.855317 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":241,"threads_started":1}
I20260811 12:51:06.321125 4104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:06.320336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.66:48357)
0811 12:51:06.320843 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:06.320937 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":269,"threads_started":1}
I20260811 12:51:06.798604 4106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:06.797913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.67:53249)
0811 12:51:06.798354 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:06.798437 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":219,"threads_started":1}
I20260811 12:51:06.857163 4107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:06.856387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.65:55271)
0811 12:51:06.856879 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:06.856961 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":250,"threads_started":1}
I20260811 12:51:07.322829 4108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:07.322094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.66:34893)
0811 12:51:07.322549 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:07.322637 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":229,"threads_started":1}
I20260811 12:51:07.800035 4109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:07.799357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.67:40597)
0811 12:51:07.799772 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:07.799852 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":204,"threads_started":1}
I20260811 12:51:07.858834 4110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:07.858108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.65:40305)
0811 12:51:07.858585 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:07.858667 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":243,"threads_started":1}
I20260811 12:51:08.324582 4111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:08.323855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.66:58841)
0811 12:51:08.324327 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:08.324409 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":231,"threads_started":1}
I20260811 12:51:08.801715 4112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:08.800942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.67:60533)
0811 12:51:08.801456 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:08.801536 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":276,"threads_started":1}
I20260811 12:51:08.860450 4114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:08.859688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.65:34175)
0811 12:51:08.860172 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:08.860271 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":246,"threads_started":1}
I20260811 12:51:09.326149 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:09.325449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.66:37871)
0811 12:51:09.325893 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:09.325985 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":216,"threads_started":1}
I20260811 12:51:09.803258 4119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:09.802510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.67:38349)
0811 12:51:09.803002 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:09.803094 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":273,"threads_started":1}
I20260811 12:51:09.862180 4120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:09.861318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.65:57149)
0811 12:51:09.861882 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:09.861964 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":224,"threads_started":1}
I20260811 12:51:10.327972 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:10.327150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.66:50267)
0811 12:51:10.327664 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:10.327762 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":214,"threads_started":1}
I20260811 12:51:10.804870 4122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:10.804102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.67:52763)
0811 12:51:10.804561 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:10.804655 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":198,"threads_started":1}
I20260811 12:51:10.863747 4123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:10.863027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.65:53763)
0811 12:51:10.863471 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:10.863561 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":232,"threads_started":1}
I20260811 12:51:11.329941 4124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:11.329053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.66:44947)
0811 12:51:11.329643 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:11.329752 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":301,"threads_started":1}
I20260811 12:51:11.806515 4125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:11.805773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.67:32985)
0811 12:51:11.806248 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:11.806341 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":232,"threads_started":1}
I20260811 12:51:11.865408 4126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:11.864614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.65:52183)
0811 12:51:11.865111 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:11.865204 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":250,"threads_started":1}
I20260811 12:51:12.331630 4127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:12.330850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.66:54603)
0811 12:51:12.331346 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:12.331456 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":243,"threads_started":1}
I20260811 12:51:12.808048 4129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:12.807299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.67:34347)
0811 12:51:12.807770 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:12.807879 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":226,"threads_started":1}
I20260811 12:51:12.867012 4130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:12.866314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.65:43079)
0811 12:51:12.866765 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:12.866845 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":239,"threads_started":1}
I20260811 12:51:13.333288 4131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:13.332542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.66:40955)
0811 12:51:13.333025 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:13.333112 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":265,"threads_started":1}
I20260811 12:51:13.809769 4134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:13.808971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.67:40833)
0811 12:51:13.809499 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:13.809595 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":269,"threads_started":1}
I20260811 12:51:13.868670 4137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:13.867926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.65:57847)
0811 12:51:13.868353 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:13.868487 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":210,"threads_started":1}
I20260811 12:51:14.335039 4138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:14.334255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.66:46567)
0811 12:51:14.334753 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:14.334853 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":228,"threads_started":1}
I20260811 12:51:14.811460 4141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:14.810701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.67:38721)
0811 12:51:14.811185 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:14.811278 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":234,"threads_started":1}
I20260811 12:51:14.870292 4143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:14.869597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.65:60461)
0811 12:51:14.870047 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:14.870130 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":236,"threads_started":1}
I20260811 12:51:15.336640 4144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:15.335951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.66:48893)
0811 12:51:15.336395 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:15.336484 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":220,"threads_started":1}
I20260811 12:51:15.812963 4145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:15.812302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.67:48941)
0811 12:51:15.812724 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:15.812805 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":214,"threads_started":1}
I20260811 12:51:15.871868 4146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:15.871107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.65:33519)
0811 12:51:15.871573 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:15.871666 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":235,"threads_started":1}
I20260811 12:51:16.338289 4148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:16.337539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.66:47407)
0811 12:51:16.337997 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:16.338095 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":224,"threads_started":1}
I20260811 12:51:16.814656 4149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:16.813876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.67:54875)
0811 12:51:16.814369 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:16.814467 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":232,"threads_started":1}
I20260811 12:51:16.873503 4150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:16.872660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.65:56771)
0811 12:51:16.873160 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:16.873293 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":237,"threads_started":1}
I20260811 12:51:17.339880 4151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:17.339163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.66:33835)
0811 12:51:17.339593 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:17.339685 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":213,"threads_started":1}
I20260811 12:51:17.816242 4152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:17.815527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.67:53521)
0811 12:51:17.815976 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:17.816053 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":232,"threads_started":1}
I20260811 12:51:17.875257 4153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:17.874556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.65:60899)
0811 12:51:17.874968 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:17.875055 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":208,"threads_started":1}
I20260811 12:51:18.341856 4154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:18.341045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.66:47105)
0811 12:51:18.341560 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:18.341654 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":258,"threads_started":1}
I20260811 12:51:18.798979 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 3373
I20260811 12:51:18.816358 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:18.800034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48800 (local address 127.7.111.65:44679)
0811 12:51:18.800678 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:18.800698 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:18.800759 (+ 61us) server_negotiation.cc:407] Connection header received
0811 12:51:18.802473 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:18.802483 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:18.802914 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:18.803204 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:18.804546 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:18.805840 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:18.807068 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:18.807731 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:18.807856 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:18.810414 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:18.810422 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:18.810436 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:18.810475 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:18.813250 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:18.814056 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:18.814064 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:18.814072 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:18.814260 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:18.814638 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:18.814641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:18.814643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:18.815157 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:18.815386 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:18.815749 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 12:51:18.816075 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":286,"threads_started":1}
I20260811 12:51:18.818041 3645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:18.818282 4158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:18.817358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40455 (local address 127.7.111.67:60605)
0811 12:51:18.817907 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:18.818059 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.126:40455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":268,"threads_started":1}
I20260811 12:51:18.900615 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 3373
I20260811 12:51:18.928062 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 3658
I20260811 12:51:18.950058 4164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:18.929000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35568 (local address 127.7.111.66:42023)
0811 12:51:18.929568 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:18.929583 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:18.929626 (+ 43us) server_negotiation.cc:407] Connection header received
0811 12:51:18.931164 (+ 1538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:18.931174 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:18.931613 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:18.931965 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:18.933666 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:18.934957 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:18.936347 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:18.937057 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:18.937250 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:18.943834 (+ 6584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:18.943843 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:18.943859 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:18.943906 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:18.946855 (+ 2949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:18.947510 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:18.947519 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:18.947525 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:18.947711 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:18.948071 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:18.948076 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:18.948079 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:18.948582 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:18.948879 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:18.949317 (+ 438us) server_negotiation.cc:299] Negotiation successful
0811 12:51:18.949763 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":225,"threads_started":1}
I20260811 12:51:18.951617 3854 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:19.034507 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 3658
I20260811 12:51:19.057337 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 3874
I20260811 12:51:19.073951 4167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:19.058337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38796 (local address 127.7.111.67:45185)
0811 12:51:19.058896 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:19.058910 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:19.058954 (+ 44us) server_negotiation.cc:407] Connection header received
0811 12:51:19.060602 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:19.060611 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:19.061043 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:19.061336 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:19.062745 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:19.063944 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:19.065145 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:19.065819 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:19.065952 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:19.068374 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:19.068382 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:19.068396 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:19.068438 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:19.071170 (+ 2732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:19.071761 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:19.071767 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:19.071773 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:19.071928 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:19.072225 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:19.072228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:19.072230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:19.072712 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:19.072935 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:19.073240 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 12:51:19.073642 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":247,"threads_started":1}
I20260811 12:51:19.075309 4076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:19.158223 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 3874
2026-08-11T12:51:19Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65210 ms)
[----------] 30 tests from SecurityITest (303261 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:51:19 dist-test-slave-f0tn krb5kdc[4173](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:51:19 dist-test-slave-f0tn krb5kdc[4173](info): set up 2 sockets
Aug 11 12:51:19 dist-test-slave-f0tn krb5kdc[4173](info): commencing operation
krb5kdc: starting...
W20260811 12:51:20.428242 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.006s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:51:20 dist-test-slave-f0tn krb5kdc[4173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:51:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:51:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
W20260811 12:51:20.868438 7613 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.7.111.126: real 0.145s user 0.000s sys 0.062s
I20260811 12:51:20.871007 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:46873
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:43629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:46873
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:51:21.601847 4195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:21.602205 4195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:21.602301 4195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:21.610097 4195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:51:21.610232 4195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:21.610280 4195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:21.610338 4195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:51:21.610383 4195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:51:21.620780 4195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:43629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:46873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:46873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:21.622323 4195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:21.623749 4195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:21.636518 4202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:21.636508 4205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:21.636546 4209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:21.637605 4195 server_base.cc:1034] running on GCE node
I20260811 12:51:21.638475 4195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:21.640044 4195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:21.641274 4195 hybrid_clock.cc:648] HybridClock initialized: now 1786452681641219 us; error 46 us; skew 500 ppm
I20260811 12:51:21.641702 4195 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:51:21 dist-test-slave-f0tn krb5kdc[4173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452681, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:51:21.644977 4195 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:51:21.646700 4195 webserver.cc:460] Webserver started at http://127.7.111.126:44745/ using document root <none> and password file <none>
I20260811 12:51:21.647429 4195 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:21.647550 4195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:21.647863 4195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:21.651017 4195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "635a3b8cfec14fb58c1c6618f7611177"
format_stamp: "Formatted at 2026-08-11 12:51:21 on dist-test-slave-f0tn"
server_key: "7d0096a83763a5acb5ca3219148e7040"
server_key_iv: "3a4ab95804cc12fddf3a659f51d8c2a5"
server_key_version: "encryptionkey@0"
I20260811 12:51:21.651796 4195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "635a3b8cfec14fb58c1c6618f7611177"
format_stamp: "Formatted at 2026-08-11 12:51:21 on dist-test-slave-f0tn"
server_key: "7d0096a83763a5acb5ca3219148e7040"
server_key_iv: "3a4ab95804cc12fddf3a659f51d8c2a5"
server_key_version: "encryptionkey@0"
I20260811 12:51:21.657343 4195 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:51:21.661087 4219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:21.662489 4195 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:51:21.662665 4195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "635a3b8cfec14fb58c1c6618f7611177"
format_stamp: "Formatted at 2026-08-11 12:51:21 on dist-test-slave-f0tn"
server_key: "7d0096a83763a5acb5ca3219148e7040"
server_key_iv: "3a4ab95804cc12fddf3a659f51d8c2a5"
server_key_version: "encryptionkey@0"
I20260811 12:51:21.662854 4195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:21.688766 4195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:21.692907 4195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:21.693197 4195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:21.711400 4195 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:46873
I20260811 12:51:21.711433 4308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:46873 every 8 connection(s)
I20260811 12:51:21.712752 4195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:51:21.714514 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 4195
I20260811 12:51:21.714745 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:51:21.715621 7613 external_mini_cluster.cc:1428] Setting key 572abc821d498f869fe018333ea45a6a
I20260811 12:51:21.717564 4309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 12:51:21 dist-test-slave-f0tn krb5kdc[4173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:51:21.731135 4309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177: Bootstrap starting.
I20260811 12:51:21.734797 4309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:21.736171 4309 log.cc:824] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:21.739379 4313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:21.717767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56458 (local address 127.7.111.126:46873)
0811 12:51:21.718531 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:21.718543 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:21.718583 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:51:21.720190 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:21.720211 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:21.720573 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:21.720904 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:21.722740 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:21.724056 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:21.725910 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:21.726559 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:21.726704 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:21.731751 (+ 5047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:21.731759 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:21.731775 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:21.731816 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:21.734845 (+ 3029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:21.735764 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:21.735772 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:21.735779 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:21.735982 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:21.736449 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:21.736453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:21.736455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:21.737229 (+ 774us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:21.737571 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:21.737960 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 12:51:21.738323 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":334,"threads_started":1}
I20260811 12:51:21.739612 4309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177: No bootstrap required, opened a new log
I20260811 12:51:21.747035 4309 raft_consensus.cc:348] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635a3b8cfec14fb58c1c6618f7611177" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46873 } }
I20260811 12:51:21.747433 4309 raft_consensus.cc:374] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:21.747534 4309 raft_consensus.cc:729] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 635a3b8cfec14fb58c1c6618f7611177, State: Initialized, Role: FOLLOWER
I20260811 12:51:21.748103 4309 consensus_queue.cc:260] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635a3b8cfec14fb58c1c6618f7611177" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46873 } }
I20260811 12:51:21.748329 4309 raft_consensus.cc:388] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:51:21.748456 4309 raft_consensus.cc:482] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:51:21.748612 4309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:21.750840 4309 raft_consensus.cc:504] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635a3b8cfec14fb58c1c6618f7611177" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46873 } }
I20260811 12:51:21.751312 4309 leader_election.cc:302] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 635a3b8cfec14fb58c1c6618f7611177; no voters:
I20260811 12:51:21.751896 4309 leader_election.cc:288] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:51:21.752017 4316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:51:21.753033 4316 raft_consensus.cc:686] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177 [term 1 LEADER]: Becoming Leader. State: Replica: 635a3b8cfec14fb58c1c6618f7611177, State: Running, Role: LEADER
I20260811 12:51:21.753705 4316 consensus_queue.cc:237] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635a3b8cfec14fb58c1c6618f7611177" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46873 } }
I20260811 12:51:21.754144 4309 sys_catalog.cc:564] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:51:21.756299 4319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 635a3b8cfec14fb58c1c6618f7611177. Latest consensus state: current_term: 1 leader_uuid: "635a3b8cfec14fb58c1c6618f7611177" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635a3b8cfec14fb58c1c6618f7611177" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46873 } } }
I20260811 12:51:21.756685 4319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177 [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:21.757263 4317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "635a3b8cfec14fb58c1c6618f7611177" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635a3b8cfec14fb58c1c6618f7611177" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46873 } } }
I20260811 12:51:21.757608 4317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177 [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:21.757793 4329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:51:21.768167 4329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:51:21.777865 4329 catalog_manager.cc:1347] Generated new cluster ID: 1fd76542ebff40d7b2aae03967915cc2
I20260811 12:51:21.777983 4329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:51:21.805761 4329 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:51:21.807765 4329 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:51:21.816875 4329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177: Generated new TSK 0
I20260811 12:51:21.817757 4329 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.111.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
I20260811 12:51:21.906548 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:46873
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:43629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:51:22.258962 4356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:22.259227 4356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:22.259359 4356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:22.267234 4356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:22.267357 4356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:22.267524 4356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:51:22.277071 4356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:43629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:46873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:22.278355 4356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:22.279711 4356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:22.292485 4365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:22.292541 4371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:22.292578 4366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:22.293350 4356 server_base.cc:1034] running on GCE node
I20260811 12:51:22.293994 4356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:22.294924 4356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:22.296128 4356 hybrid_clock.cc:648] HybridClock initialized: now 1786452682296068 us; error 48 us; skew 500 ppm
I20260811 12:51:22.296439 4356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:51:22 dist-test-slave-f0tn krb5kdc[4173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452682, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:51:22.299513 4356 init.cc:377] Logged in from keytab as kudu/127.7.111.65@KRBTEST.COM (short username kudu)
I20260811 12:51:22.300879 4356 webserver.cc:460] Webserver started at http://127.7.111.65:40539/ using document root <none> and password file <none>
I20260811 12:51:22.301604 4356 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:22.301720 4356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:22.302028 4356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:22.305262 4356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "f98aa606fbbc4fef924531df98ab46ee"
format_stamp: "Formatted at 2026-08-11 12:51:22 on dist-test-slave-f0tn"
server_key: "74902da25df210ce503350ee5a8b4b32"
server_key_iv: "1e7e25ebb4b8050ebc2a6dcf15be062c"
server_key_version: "encryptionkey@0"
I20260811 12:51:22.306037 4356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "f98aa606fbbc4fef924531df98ab46ee"
format_stamp: "Formatted at 2026-08-11 12:51:22 on dist-test-slave-f0tn"
server_key: "74902da25df210ce503350ee5a8b4b32"
server_key_iv: "1e7e25ebb4b8050ebc2a6dcf15be062c"
server_key_version: "encryptionkey@0"
I20260811 12:51:22.311017 4356 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:51:22.314486 4381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:22.315811 4356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:51:22.315994 4356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "f98aa606fbbc4fef924531df98ab46ee"
format_stamp: "Formatted at 2026-08-11 12:51:22 on dist-test-slave-f0tn"
server_key: "74902da25df210ce503350ee5a8b4b32"
server_key_iv: "1e7e25ebb4b8050ebc2a6dcf15be062c"
server_key_version: "encryptionkey@0"
I20260811 12:51:22.316205 4356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:22.347160 4356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:22.351537 4356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:22.351797 4356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:22.352772 4356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:51:22.354118 4356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:51:22.354194 4356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:22.354301 4356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:51:22.354370 4356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:22.385403 4356 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:33041
I20260811 12:51:22.385474 4586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:33041 every 8 connection(s)
I20260811 12:51:22.386699 4356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:51:22.394960 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 4356
I20260811 12:51:22.395177 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:51:22.395802 7613 external_mini_cluster.cc:1428] Setting key 5eba078877d83ae47a197ac470a16118
Aug 11 12:51:22 dist-test-slave-f0tn krb5kdc[4173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452682, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:51:22.406425 4590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:22.389243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:43583 (local address 127.7.111.126:46873)
0811 12:51:22.389715 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:22.389719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:22.390600 (+ 881us) server_negotiation.cc:407] Connection header received
0811 12:51:22.392431 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:22.392437 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:22.392529 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:22.392627 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:22.394171 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:22.394895 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:22.396528 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:22.397029 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:22.397184 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:22.400539 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:22.400545 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:22.400549 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:22.400587 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:22.403485 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:22.404154 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:22.404159 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:22.404162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:22.404307 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:22.404708 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:22.404712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:22.404715 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:22.405032 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:22.405167 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:22.405867 (+ 700us) server_negotiation.cc:299] Negotiation successful
0811 12:51:22.406088 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":246,"threads_started":1}
I20260811 12:51:22.407255 4591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:22.389748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:46873 (local address 127.7.111.65:43583)
0811 12:51:22.390436 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:22.390481 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:22.392154 (+ 1673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:22.392869 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:22.392878 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:22.393430 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:51:22.393997 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:22.394015 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:22.395060 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:22.395064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:22.396384 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:22.396397 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:22.397262 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:22.397268 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:22.397560 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:22.398226 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:51:22.398249 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:51:22.400323 (+ 2074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:51:22.403614 (+ 3291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:51:22.403621 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:51:22.403635 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:51:22.404032 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:51:22.404398 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:51:22.404401 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:51:22.404404 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:51:22.404595 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:51:22.405130 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:51:22.405135 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:51:22.405488 (+ 353us) client_negotiation.cc:769] Sending connection context
0811 12:51:22.405730 (+ 242us) client_negotiation.cc:240] Negotiation successful
0811 12:51:22.406124 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":269,"threads_started":1}
I20260811 12:51:22.408826 4587 heartbeater.cc:347] Connected to a master server at 127.7.111.126:46873
I20260811 12:51:22.409121 4587 heartbeater.cc:464] Registering TS with master...
I20260811 12:51:22.409873 4587 heartbeater.cc:511] Master 127.7.111.126:46873 requested a full tablet report, sending...
I20260811 12:51:22.411887 4244 ts_manager.cc:194] Registered new tserver with Master: f98aa606fbbc4fef924531df98ab46ee (127.7.111.65:33041)
I20260811 12:51:22.413823 4244 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:43583
WARNING: no policy specified for kudu/127.7.111.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
I20260811 12:51:22.465857 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:46873
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:43629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:51:22.818250 4597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:22.818534 4597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:22.818674 4597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:22.826069 4597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:22.826197 4597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:22.826341 4597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:51:22.836071 4597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:43629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:46873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:22.837306 4597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:22.838620 4597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:22.851063 4610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:22.851122 4608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:22.851145 4612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:22.851891 4597 server_base.cc:1034] running on GCE node
I20260811 12:51:22.852442 4597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:22.853279 4597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:22.854471 4597 hybrid_clock.cc:648] HybridClock initialized: now 1786452682854432 us; error 62 us; skew 500 ppm
I20260811 12:51:22.854780 4597 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:51:22 dist-test-slave-f0tn krb5kdc[4173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452682, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:51:22.857937 4597 init.cc:377] Logged in from keytab as kudu/127.7.111.66@KRBTEST.COM (short username kudu)
I20260811 12:51:22.859265 4597 webserver.cc:460] Webserver started at http://127.7.111.66:36349/ using document root <none> and password file <none>
I20260811 12:51:22.859900 4597 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:22.860019 4597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:22.860329 4597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:22.863338 4597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "85aa12e8ca264e318539cd3eca1ebae4"
format_stamp: "Formatted at 2026-08-11 12:51:22 on dist-test-slave-f0tn"
server_key: "b1d2704b6cb880cb079ba5cbd265c2d5"
server_key_iv: "14e86ed424bc938b25fa7157836dfd5f"
server_key_version: "encryptionkey@0"
I20260811 12:51:22.864116 4597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "85aa12e8ca264e318539cd3eca1ebae4"
format_stamp: "Formatted at 2026-08-11 12:51:22 on dist-test-slave-f0tn"
server_key: "b1d2704b6cb880cb079ba5cbd265c2d5"
server_key_iv: "14e86ed424bc938b25fa7157836dfd5f"
server_key_version: "encryptionkey@0"
I20260811 12:51:22.869035 4597 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 12:51:22.872290 4620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:22.873557 4597 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:51:22.873710 4597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "85aa12e8ca264e318539cd3eca1ebae4"
format_stamp: "Formatted at 2026-08-11 12:51:22 on dist-test-slave-f0tn"
server_key: "b1d2704b6cb880cb079ba5cbd265c2d5"
server_key_iv: "14e86ed424bc938b25fa7157836dfd5f"
server_key_version: "encryptionkey@0"
I20260811 12:51:22.873885 4597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:22.907532 4597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:22.911491 4597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:22.911761 4597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:22.912714 4597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:51:22.914083 4597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:51:22.914165 4597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:22.914302 4597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:51:22.914356 4597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:22.944202 4597 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:46107
I20260811 12:51:22.944247 4807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:46107 every 8 connection(s)
I20260811 12:51:22.945534 4597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:51:22.947542 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 4597
I20260811 12:51:22.947738 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:51:22.948235 7613 external_mini_cluster.cc:1428] Setting key 9bf85a614692aae12db18fe1f84fe8ff
Aug 11 12:51:22 dist-test-slave-f0tn krb5kdc[4173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452682, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:51:22.967725 4814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:22.948435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:46359 (local address 127.7.111.126:46873)
0811 12:51:22.948904 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:22.948910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:22.950058 (+ 1148us) server_negotiation.cc:407] Connection header received
0811 12:51:22.952434 (+ 2376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:22.952438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:22.952501 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:22.952590 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:22.954469 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:22.955236 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:22.957226 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:22.957690 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:22.957784 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:22.961725 (+ 3941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:22.961730 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:22.961733 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:22.961777 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:22.964609 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:22.965336 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:22.965341 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:22.965344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:22.965523 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:22.965941 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:22.965946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:22.965949 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:22.966274 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:22.966403 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:22.967269 (+ 866us) server_negotiation.cc:299] Negotiation successful
0811 12:51:22.967458 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":212,"threads_started":1}
I20260811 12:51:22.969094 4816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:22.948998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:46873 (local address 127.7.111.66:46359)
0811 12:51:22.949881 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:22.949943 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:22.952141 (+ 2198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:22.952847 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:22.952857 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:22.953584 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:51:22.954326 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:22.954349 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:22.955390 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:22.955395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:22.957100 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:22.957115 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:22.957905 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:22.957913 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:22.958173 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:22.958874 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:51:22.958909 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:51:22.961489 (+ 2580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:51:22.964750 (+ 3261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:51:22.964758 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:51:22.964770 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:51:22.965211 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:51:22.965612 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:51:22.965616 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:51:22.965618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:51:22.965810 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:51:22.966401 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:51:22.966409 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:51:22.966859 (+ 450us) client_negotiation.cc:769] Sending connection context
0811 12:51:22.967184 (+ 325us) client_negotiation.cc:240] Negotiation successful
0811 12:51:22.967727 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":376,"threads_started":1}
I20260811 12:51:22.970503 4810 heartbeater.cc:347] Connected to a master server at 127.7.111.126:46873
I20260811 12:51:22.970810 4810 heartbeater.cc:464] Registering TS with master...
I20260811 12:51:22.971522 4810 heartbeater.cc:511] Master 127.7.111.126:46873 requested a full tablet report, sending...
I20260811 12:51:22.972896 4244 ts_manager.cc:194] Registered new tserver with Master: 85aa12e8ca264e318539cd3eca1ebae4 (127.7.111.66:46107)
I20260811 12:51:22.973760 4244 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:46359
WARNING: no policy specified for kudu/127.7.111.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
I20260811 12:51:23.020117 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:46873
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:43629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:51:23.361069 4821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:23.361409 4821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:23.361560 4821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:23.369168 4821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:23.369290 4821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:23.369478 4821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:51:23.379542 4821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:43629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:46873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:23.380934 4821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:23.382392 4821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:23.397472 4831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:23.397431 4836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:23.397495 4833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:23.398589 4821 server_base.cc:1034] running on GCE node
I20260811 12:51:23.399264 4821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:23.400266 4821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:23.401472 4821 hybrid_clock.cc:648] HybridClock initialized: now 1786452683401412 us; error 51 us; skew 500 ppm
I20260811 12:51:23.401818 4821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:51:23 dist-test-slave-f0tn krb5kdc[4173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452683, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:51:23.404971 4821 init.cc:377] Logged in from keytab as kudu/127.7.111.67@KRBTEST.COM (short username kudu)
I20260811 12:51:23.406430 4821 webserver.cc:460] Webserver started at http://127.7.111.67:39743/ using document root <none> and password file <none>
I20260811 12:51:23.407136 4821 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:23.407249 4821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:23.407517 4821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:23.410555 4821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "ddda3c5ab68642a98b3961e254dc675d"
format_stamp: "Formatted at 2026-08-11 12:51:23 on dist-test-slave-f0tn"
server_key: "d529fe9f5e3921fce0b185b923c48886"
server_key_iv: "809eba463dd5737f8dc62246414d8456"
server_key_version: "encryptionkey@0"
I20260811 12:51:23.411336 4821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "ddda3c5ab68642a98b3961e254dc675d"
format_stamp: "Formatted at 2026-08-11 12:51:23 on dist-test-slave-f0tn"
server_key: "d529fe9f5e3921fce0b185b923c48886"
server_key_iv: "809eba463dd5737f8dc62246414d8456"
server_key_version: "encryptionkey@0"
I20260811 12:51:23.416550 4821 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:51:23.416690 4587 heartbeater.cc:503] Master 127.7.111.126:46873 was elected leader, sending a full tablet report...
I20260811 12:51:23.420571 4848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:23.422072 4821 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:51:23.422247 4821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "ddda3c5ab68642a98b3961e254dc675d"
format_stamp: "Formatted at 2026-08-11 12:51:23 on dist-test-slave-f0tn"
server_key: "d529fe9f5e3921fce0b185b923c48886"
server_key_iv: "809eba463dd5737f8dc62246414d8456"
server_key_version: "encryptionkey@0"
I20260811 12:51:23.422461 4821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:23.445238 4821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:23.449922 4821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:23.450224 4821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:23.451285 4821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:51:23.452719 4821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:51:23.452809 4821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:23.452941 4821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:51:23.452998 4821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:23.485283 4821 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:41629
I20260811 12:51:23.485325 5042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:41629 every 8 connection(s)
I20260811 12:51:23.486588 4821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:51:23.491466 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 4821
I20260811 12:51:23.491663 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:51:23.492231 7613 external_mini_cluster.cc:1428] Setting key ff03d4b574130bd6ca9baf9309eea2ac
Aug 11 12:51:23 dist-test-slave-f0tn krb5kdc[4173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452683, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:51:23.507571 5045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:23.489082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:52533 (local address 127.7.111.126:46873)
0811 12:51:23.489522 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:23.489527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:23.490503 (+ 976us) server_negotiation.cc:407] Connection header received
0811 12:51:23.492946 (+ 2443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:23.492952 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:23.493037 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:23.493137 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:23.494870 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.495751 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.497508 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.497915 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.498028 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:23.501426 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:23.501432 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:23.501436 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:23.501471 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:23.504039 (+ 2568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:23.504851 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:23.504857 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:23.504861 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:23.505018 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:23.505544 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:23.505550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:23.505553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:23.505848 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:23.506002 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:23.506981 (+ 979us) server_negotiation.cc:299] Negotiation successful
0811 12:51:23.507204 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":212,"threads_started":1}
I20260811 12:51:23.508497 5046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:23.489594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:46873 (local address 127.7.111.67:52533)
0811 12:51:23.490337 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:23.490376 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:23.492618 (+ 2242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:23.493419 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:23.493431 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:23.494118 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:51:23.494692 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:23.494721 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.495901 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.495905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:23.497327 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:23.497341 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.498153 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.498162 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:23.498380 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:23.498988 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:51:23.499012 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:51:23.501162 (+ 2150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:51:23.504183 (+ 3021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:51:23.504193 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:51:23.504210 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:51:23.504724 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:51:23.505147 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:51:23.505153 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:51:23.505157 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:51:23.505427 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:51:23.506082 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:51:23.506090 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:51:23.506496 (+ 406us) client_negotiation.cc:769] Sending connection context
0811 12:51:23.506814 (+ 318us) client_negotiation.cc:240] Negotiation successful
0811 12:51:23.507273 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":266,"threads_started":1}
I20260811 12:51:23.509850 5043 heartbeater.cc:347] Connected to a master server at 127.7.111.126:46873
I20260811 12:51:23.510210 5043 heartbeater.cc:464] Registering TS with master...
I20260811 12:51:23.510980 5043 heartbeater.cc:511] Master 127.7.111.126:46873 requested a full tablet report, sending...
I20260811 12:51:23.512518 4244 ts_manager.cc:194] Registered new tserver with Master: ddda3c5ab68642a98b3961e254dc675d (127.7.111.67:41629)
I20260811 12:51:23.513334 4244 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:52533
I20260811 12:51:23.520248 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:51:23.536795 5045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:23.524870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56464 (local address 127.7.111.126:46873)
0811 12:51:23.525020 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:23.525024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:23.525493 (+ 469us) server_negotiation.cc:407] Connection header received
0811 12:51:23.525682 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:23.525686 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:23.525752 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:23.525823 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:23.527256 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.528028 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.529340 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.529745 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.529834 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:23.532199 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:23.532214 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:23.532217 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:23.532243 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:23.534598 (+ 2355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:23.535215 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:23.535218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:23.535221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:23.535332 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:23.535659 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:23.535661 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:23.535663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:23.535955 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:23.536133 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:23.536403 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 12:51:23.536564 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 12:51:23.540838 4244 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56464:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:51:23.547910 4244 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:51:23.586833 5075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:23.562065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47912 (local address 127.7.111.67:41629)
0811 12:51:23.568933 (+ 6868us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:23.568942 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:23.577680 (+ 8738us) server_negotiation.cc:407] Connection header received
0811 12:51:23.578013 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:23.578021 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:23.578349 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:23.578526 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:23.579460 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.581764 (+ 2304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.584261 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.585691 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.585820 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:23.586101 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:23.586308 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 12:51:23.586485 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6679,"thread_start_us":6473,"threads_started":1}
I20260811 12:51:23.586875 5068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:23.560649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:41629 (local address 127.0.0.1:47912)
0811 12:51:23.577541 (+ 16892us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:23.577564 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:23.577809 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:23.578718 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:23.578725 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:23.578756 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:23.579229 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:23.579250 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.582047 (+ 2797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.582052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:23.584098 (+ 2046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:23.584111 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.585879 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.585886 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:23.586064 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:23.586116 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 12:51:23.586255 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 12:51:23.586452 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16769,"mutex_wait_us":55,"thread_start_us":16585,"threads_started":1}
I20260811 12:51:23.588192 5070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:23.560418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:46107 (local address 127.0.0.1:45650)
0811 12:51:23.575792 (+ 15374us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:23.575873 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:23.576139 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:23.580248 (+ 4109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:23.580252 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:23.580280 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:23.580794 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:23.580809 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.582674 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.582680 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:23.585042 (+ 2362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:23.585063 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.587282 (+ 2219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.587289 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:23.587520 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:23.587553 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:51:23.587644 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 12:51:23.587794 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15028,"mutex_wait_us":38,"thread_start_us":18553,"threads_started":1}
I20260811 12:51:23.588232 5066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:23.560477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45650 (local address 127.7.111.66:46107)
0811 12:51:23.579681 (+ 19204us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:23.579689 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:23.579717 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:51:23.579799 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:23.579805 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:23.579999 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:23.580821 (+ 822us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:23.581066 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.582520 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.585249 (+ 2729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.586742 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.586872 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:23.587457 (+ 585us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:23.587710 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 12:51:23.587881 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19040,"thread_start_us":18934,"threads_started":1}
I20260811 12:51:23.588745 5067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:23.560858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:33041 (local address 127.0.0.1:35308)
0811 12:51:23.579299 (+ 18441us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:23.579321 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:23.579538 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:23.580220 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:23.580225 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:23.580252 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:23.580953 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:23.580961 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.582887 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.582893 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:23.585097 (+ 2204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:23.585107 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.587304 (+ 2197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.587311 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:23.587542 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:23.587569 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 12:51:23.587645 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 12:51:23.588355 (+ 710us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18283,"mutex_wait_us":600,"spinlock_wait_cycles":1408,"thread_start_us":15520,"threads_started":1}
I20260811 12:51:23.589250 5073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:23.561701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35308 (local address 127.7.111.65:33041)
0811 12:51:23.569161 (+ 7460us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:23.569171 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:23.579429 (+ 10258us) server_negotiation.cc:407] Connection header received
0811 12:51:23.579765 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:23.579773 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:23.580011 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:23.580221 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:23.581138 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.582743 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.585269 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.586734 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.586879 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:23.587107 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:23.587775 (+ 668us) server_negotiation.cc:299] Negotiation successful
0811 12:51:23.587934 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7240,"thread_start_us":7072,"threads_started":1}
I20260811 12:51:23.591254 4933 tablet_service.cc:1467] Processing CreateTablet for tablet 57287328975f4ecf835d01fc0cc143f2 (DEFAULT_TABLE table=test-table [id=d161fa40ff7d4951904737be4cd8c71d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:51:23.592567 4933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57287328975f4ecf835d01fc0cc143f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:23.592211 4691 tablet_service.cc:1467] Processing CreateTablet for tablet 57287328975f4ecf835d01fc0cc143f2 (DEFAULT_TABLE table=test-table [id=d161fa40ff7d4951904737be4cd8c71d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:51:23.592700 4476 tablet_service.cc:1467] Processing CreateTablet for tablet 57287328975f4ecf835d01fc0cc143f2 (DEFAULT_TABLE table=test-table [id=d161fa40ff7d4951904737be4cd8c71d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:51:23.593689 4691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57287328975f4ecf835d01fc0cc143f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:23.594132 4476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57287328975f4ecf835d01fc0cc143f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:23.604339 5077 tablet_bootstrap.cc:492] T 57287328975f4ecf835d01fc0cc143f2 P ddda3c5ab68642a98b3961e254dc675d: Bootstrap starting.
I20260811 12:51:23.608444 5077 tablet_bootstrap.cc:654] T 57287328975f4ecf835d01fc0cc143f2 P ddda3c5ab68642a98b3961e254dc675d: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:23.608453 5078 tablet_bootstrap.cc:492] T 57287328975f4ecf835d01fc0cc143f2 P f98aa606fbbc4fef924531df98ab46ee: Bootstrap starting.
I20260811 12:51:23.609982 5077 log.cc:824] T 57287328975f4ecf835d01fc0cc143f2 P ddda3c5ab68642a98b3961e254dc675d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:23.611518 5079 tablet_bootstrap.cc:492] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4: Bootstrap starting.
I20260811 12:51:23.612191 5078 tablet_bootstrap.cc:654] T 57287328975f4ecf835d01fc0cc143f2 P f98aa606fbbc4fef924531df98ab46ee: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:23.613611 5078 log.cc:824] T 57287328975f4ecf835d01fc0cc143f2 P f98aa606fbbc4fef924531df98ab46ee: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:23.615098 5079 tablet_bootstrap.cc:654] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:23.616348 5079 log.cc:824] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:23.616863 5078 tablet_bootstrap.cc:492] T 57287328975f4ecf835d01fc0cc143f2 P f98aa606fbbc4fef924531df98ab46ee: No bootstrap required, opened a new log
I20260811 12:51:23.617168 5078 ts_tablet_manager.cc:1397] T 57287328975f4ecf835d01fc0cc143f2 P f98aa606fbbc4fef924531df98ab46ee: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 12:51:23.618780 5077 tablet_bootstrap.cc:492] T 57287328975f4ecf835d01fc0cc143f2 P ddda3c5ab68642a98b3961e254dc675d: No bootstrap required, opened a new log
I20260811 12:51:23.619059 5077 ts_tablet_manager.cc:1397] T 57287328975f4ecf835d01fc0cc143f2 P ddda3c5ab68642a98b3961e254dc675d: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.000s
I20260811 12:51:23.619170 5079 tablet_bootstrap.cc:492] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4: No bootstrap required, opened a new log
I20260811 12:51:23.619421 5079 ts_tablet_manager.cc:1397] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 12:51:23.626920 5079 raft_consensus.cc:348] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85aa12e8ca264e318539cd3eca1ebae4" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46107 } } peers { permanent_uuid: "ddda3c5ab68642a98b3961e254dc675d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41629 } } peers { permanent_uuid: "f98aa606fbbc4fef924531df98ab46ee" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33041 } }
I20260811 12:51:23.627305 5079 raft_consensus.cc:374] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:23.627416 5079 raft_consensus.cc:729] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85aa12e8ca264e318539cd3eca1ebae4, State: Initialized, Role: FOLLOWER
I20260811 12:51:23.627799 5078 raft_consensus.cc:348] T 57287328975f4ecf835d01fc0cc143f2 P f98aa606fbbc4fef924531df98ab46ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85aa12e8ca264e318539cd3eca1ebae4" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46107 } } peers { permanent_uuid: "ddda3c5ab68642a98b3961e254dc675d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41629 } } peers { permanent_uuid: "f98aa606fbbc4fef924531df98ab46ee" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33041 } }
I20260811 12:51:23.628012 5079 consensus_queue.cc:260] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85aa12e8ca264e318539cd3eca1ebae4" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46107 } } peers { permanent_uuid: "ddda3c5ab68642a98b3961e254dc675d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41629 } } peers { permanent_uuid: "f98aa606fbbc4fef924531df98ab46ee" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33041 } }
I20260811 12:51:23.628238 5078 raft_consensus.cc:374] T 57287328975f4ecf835d01fc0cc143f2 P f98aa606fbbc4fef924531df98ab46ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:23.628340 5078 raft_consensus.cc:729] T 57287328975f4ecf835d01fc0cc143f2 P f98aa606fbbc4fef924531df98ab46ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f98aa606fbbc4fef924531df98ab46ee, State: Initialized, Role: FOLLOWER
I20260811 12:51:23.628926 5078 consensus_queue.cc:260] T 57287328975f4ecf835d01fc0cc143f2 P f98aa606fbbc4fef924531df98ab46ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85aa12e8ca264e318539cd3eca1ebae4" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46107 } } peers { permanent_uuid: "ddda3c5ab68642a98b3961e254dc675d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41629 } } peers { permanent_uuid: "f98aa606fbbc4fef924531df98ab46ee" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33041 } }
I20260811 12:51:23.629571 5077 raft_consensus.cc:348] T 57287328975f4ecf835d01fc0cc143f2 P ddda3c5ab68642a98b3961e254dc675d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85aa12e8ca264e318539cd3eca1ebae4" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46107 } } peers { permanent_uuid: "ddda3c5ab68642a98b3961e254dc675d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41629 } } peers { permanent_uuid: "f98aa606fbbc4fef924531df98ab46ee" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33041 } }
I20260811 12:51:23.630003 5077 raft_consensus.cc:374] T 57287328975f4ecf835d01fc0cc143f2 P ddda3c5ab68642a98b3961e254dc675d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:23.630110 5077 raft_consensus.cc:729] T 57287328975f4ecf835d01fc0cc143f2 P ddda3c5ab68642a98b3961e254dc675d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddda3c5ab68642a98b3961e254dc675d, State: Initialized, Role: FOLLOWER
I20260811 12:51:23.630666 5077 consensus_queue.cc:260] T 57287328975f4ecf835d01fc0cc143f2 P ddda3c5ab68642a98b3961e254dc675d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85aa12e8ca264e318539cd3eca1ebae4" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46107 } } peers { permanent_uuid: "ddda3c5ab68642a98b3961e254dc675d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41629 } } peers { permanent_uuid: "f98aa606fbbc4fef924531df98ab46ee" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33041 } }
I20260811 12:51:23.631814 5043 heartbeater.cc:503] Master 127.7.111.126:46873 was elected leader, sending a full tablet report...
I20260811 12:51:23.632447 5077 ts_tablet_manager.cc:1428] T 57287328975f4ecf835d01fc0cc143f2 P ddda3c5ab68642a98b3961e254dc675d: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260811 12:51:23.633330 4810 heartbeater.cc:503] Master 127.7.111.126:46873 was elected leader, sending a full tablet report...
I20260811 12:51:23.634037 5079 ts_tablet_manager.cc:1428] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
I20260811 12:51:23.636109 5078 ts_tablet_manager.cc:1428] T 57287328975f4ecf835d01fc0cc143f2 P f98aa606fbbc4fef924531df98ab46ee: Time spent starting tablet: real 0.019s user 0.012s sys 0.003s
W20260811 12:51:23.638934 4589 tablet.cc:2369] T 57287328975f4ecf835d01fc0cc143f2 P f98aa606fbbc4fef924531df98ab46ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:51:23.697480 4811 tablet.cc:2369] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:51:23.738161 5044 tablet.cc:2369] T 57287328975f4ecf835d01fc0cc143f2 P ddda3c5ab68642a98b3961e254dc675d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:51:23.742555 5084 raft_consensus.cc:482] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:51:23.742841 5084 raft_consensus.cc:504] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85aa12e8ca264e318539cd3eca1ebae4" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46107 } } peers { permanent_uuid: "ddda3c5ab68642a98b3961e254dc675d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41629 } } peers { permanent_uuid: "f98aa606fbbc4fef924531df98ab46ee" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33041 } }
I20260811 12:51:23.744524 5084 leader_election.cc:288] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ddda3c5ab68642a98b3961e254dc675d (127.7.111.67:41629), f98aa606fbbc4fef924531df98ab46ee (127.7.111.65:33041)
I20260811 12:51:23.751080 5073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:23.744972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:48501 (local address 127.7.111.65:33041)
0811 12:51:23.745165 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:23.745171 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:23.746003 (+ 832us) server_negotiation.cc:407] Connection header received
0811 12:51:23.746162 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:23.746166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:23.746235 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:23.746310 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:23.746961 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.747666 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.749460 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.750196 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.750352 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:23.750428 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:23.750719 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 12:51:23.750844 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:51:23.751153 5075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:23.744830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:43285 (local address 127.7.111.67:41629)
0811 12:51:23.745045 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:23.745052 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:23.745923 (+ 871us) server_negotiation.cc:407] Connection header received
0811 12:51:23.746115 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:23.746119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:23.746190 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:23.746265 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:23.746869 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.747607 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.749488 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.750270 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.750420 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:23.750494 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:23.750786 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 12:51:23.750919 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 12:51:23.751272 5090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:23.745072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:33041 (local address 127.7.111.66:48501)
0811 12:51:23.745886 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:23.745901 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:23.746028 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:23.746339 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:23.746343 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:23.746366 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:23.746800 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:23.746809 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.747798 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.747802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:23.749281 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:23.749294 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.750429 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.750436 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:23.750569 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:23.750592 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 12:51:23.750672 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 12:51:23.750824 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"mutex_wait_us":4,"thread_start_us":465,"threads_started":1}
I20260811 12:51:23.751315 5089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:23.744931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:41629 (local address 127.7.111.66:43285)
0811 12:51:23.745813 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:23.745830 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:23.745980 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:23.746295 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:23.746300 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:23.746331 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:23.746709 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:23.746718 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.747742 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.747748 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:23.749320 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:23.749333 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.750491 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.750501 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:23.750626 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:23.750648 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 12:51:23.750736 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 12:51:23.750889 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"mutex_wait_us":73,"thread_start_us":501,"threads_started":1}
I20260811 12:51:23.752223 4972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57287328975f4ecf835d01fc0cc143f2" candidate_uuid: "85aa12e8ca264e318539cd3eca1ebae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddda3c5ab68642a98b3961e254dc675d" is_pre_election: true
I20260811 12:51:23.752223 4517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57287328975f4ecf835d01fc0cc143f2" candidate_uuid: "85aa12e8ca264e318539cd3eca1ebae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f98aa606fbbc4fef924531df98ab46ee" is_pre_election: true
I20260811 12:51:23.752672 4517 raft_consensus.cc:2454] T 57287328975f4ecf835d01fc0cc143f2 P f98aa606fbbc4fef924531df98ab46ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85aa12e8ca264e318539cd3eca1ebae4 in term 0.
I20260811 12:51:23.752693 4972 raft_consensus.cc:2454] T 57287328975f4ecf835d01fc0cc143f2 P ddda3c5ab68642a98b3961e254dc675d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85aa12e8ca264e318539cd3eca1ebae4 in term 0.
I20260811 12:51:23.753290 4629 leader_election.cc:302] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85aa12e8ca264e318539cd3eca1ebae4, f98aa606fbbc4fef924531df98ab46ee; no voters:
I20260811 12:51:23.753700 5084 raft_consensus.cc:2781] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:51:23.753855 5084 raft_consensus.cc:482] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:51:23.753960 5084 raft_consensus.cc:3037] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:23.756212 5084 raft_consensus.cc:504] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85aa12e8ca264e318539cd3eca1ebae4" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46107 } } peers { permanent_uuid: "ddda3c5ab68642a98b3961e254dc675d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41629 } } peers { permanent_uuid: "f98aa606fbbc4fef924531df98ab46ee" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33041 } }
I20260811 12:51:23.756736 5084 leader_election.cc:288] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4 [CANDIDATE]: Term 1 election: Requested vote from peers ddda3c5ab68642a98b3961e254dc675d (127.7.111.67:41629), f98aa606fbbc4fef924531df98ab46ee (127.7.111.65:33041)
I20260811 12:51:23.757218 4517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57287328975f4ecf835d01fc0cc143f2" candidate_uuid: "85aa12e8ca264e318539cd3eca1ebae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f98aa606fbbc4fef924531df98ab46ee"
I20260811 12:51:23.757218 4972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57287328975f4ecf835d01fc0cc143f2" candidate_uuid: "85aa12e8ca264e318539cd3eca1ebae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddda3c5ab68642a98b3961e254dc675d"
I20260811 12:51:23.757450 4972 raft_consensus.cc:3037] T 57287328975f4ecf835d01fc0cc143f2 P ddda3c5ab68642a98b3961e254dc675d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:23.757450 4517 raft_consensus.cc:3037] T 57287328975f4ecf835d01fc0cc143f2 P f98aa606fbbc4fef924531df98ab46ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:23.760565 4517 raft_consensus.cc:2454] T 57287328975f4ecf835d01fc0cc143f2 P f98aa606fbbc4fef924531df98ab46ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85aa12e8ca264e318539cd3eca1ebae4 in term 1.
I20260811 12:51:23.760571 4972 raft_consensus.cc:2454] T 57287328975f4ecf835d01fc0cc143f2 P ddda3c5ab68642a98b3961e254dc675d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85aa12e8ca264e318539cd3eca1ebae4 in term 1.
I20260811 12:51:23.760968 4624 leader_election.cc:302] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85aa12e8ca264e318539cd3eca1ebae4, ddda3c5ab68642a98b3961e254dc675d; no voters:
I20260811 12:51:23.761250 5084 raft_consensus.cc:2781] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:51:23.761679 5084 raft_consensus.cc:686] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4 [term 1 LEADER]: Becoming Leader. State: Replica: 85aa12e8ca264e318539cd3eca1ebae4, State: Running, Role: LEADER
I20260811 12:51:23.762173 5084 consensus_queue.cc:237] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85aa12e8ca264e318539cd3eca1ebae4" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46107 } } peers { permanent_uuid: "ddda3c5ab68642a98b3961e254dc675d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41629 } } peers { permanent_uuid: "f98aa606fbbc4fef924531df98ab46ee" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33041 } }
I20260811 12:51:23.766501 4244 catalog_manager.cc:5495] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 85aa12e8ca264e318539cd3eca1ebae4 (127.7.111.66). New cstate: current_term: 1 leader_uuid: "85aa12e8ca264e318539cd3eca1ebae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85aa12e8ca264e318539cd3eca1ebae4" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 46107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ddda3c5ab68642a98b3961e254dc675d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f98aa606fbbc4fef924531df98ab46ee" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33041 } health_report { overall_health: UNKNOWN } } }
I20260811 12:51:23.784515 5066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:23.779336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45658 (local address 127.7.111.66:46107)
0811 12:51:23.779513 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:23.779519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:23.779536 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:51:23.779737 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:23.779741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:23.779824 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:23.779950 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:51:23.780332 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.781029 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.782411 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.782845 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.782956 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:23.783431 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:51:23.783939 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:51:23.784069 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:23.784176 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 12:51:23.784255 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 12:51:23.792284 4972 raft_consensus.cc:1261] T 57287328975f4ecf835d01fc0cc143f2 P ddda3c5ab68642a98b3961e254dc675d [term 1 FOLLOWER]: Refusing update from remote peer 85aa12e8ca264e318539cd3eca1ebae4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:51:23.792593 4517 raft_consensus.cc:1261] T 57287328975f4ecf835d01fc0cc143f2 P f98aa606fbbc4fef924531df98ab46ee [term 1 FOLLOWER]: Refusing update from remote peer 85aa12e8ca264e318539cd3eca1ebae4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:51:23.793315 5092 consensus_queue.cc:1035] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddda3c5ab68642a98b3961e254dc675d" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:51:23.793689 5084 consensus_queue.cc:1035] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f98aa606fbbc4fef924531df98ab46ee" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 33041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:51:23.803479 5097 mvcc.cc:205] Tried to move back new op lower bound from 7317310192803491840 to 7317310192693497856. Current Snapshot: MvccSnapshot[applied={T|T < 7317310192803491840}]
I20260811 12:51:23.805012 5102 mvcc.cc:205] Tried to move back new op lower bound from 7317310192803491840 to 7317310192693497856. Current Snapshot: MvccSnapshot[applied={T|T < 7317310192803491840}]
I20260811 12:51:23.809413 5100 mvcc.cc:205] Tried to move back new op lower bound from 7317310192803491840 to 7317310192693497856. Current Snapshot: MvccSnapshot[applied={T|T < 7317310192803491840}]
I20260811 12:51:23.817059 4244 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:51:23.817277 4244 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:51:23.820714 4244 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 635a3b8cfec14fb58c1c6618f7611177: Sending DeleteTablet for 3 replicas of tablet 57287328975f4ecf835d01fc0cc143f2
I20260811 12:51:23.821934 4933 tablet_service.cc:1514] Processing DeleteTablet for tablet 57287328975f4ecf835d01fc0cc143f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:51:23 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:47912
I20260811 12:51:23.821919 4691 tablet_service.cc:1514] Processing DeleteTablet for tablet 57287328975f4ecf835d01fc0cc143f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:51:23 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:45650
I20260811 12:51:23.822325 4476 tablet_service.cc:1514] Processing DeleteTablet for tablet 57287328975f4ecf835d01fc0cc143f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:51:23 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:35308
I20260811 12:51:23.826994 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 4356
I20260811 12:51:23.840826 5115 tablet_replica.cc:330] T 57287328975f4ecf835d01fc0cc143f2 P ddda3c5ab68642a98b3961e254dc675d: stopping tablet replica
I20260811 12:51:23.843287 5116 tablet_replica.cc:330] T 57287328975f4ecf835d01fc0cc143f2 P f98aa606fbbc4fef924531df98ab46ee: stopping tablet replica
I20260811 12:51:23.843523 5114 tablet_replica.cc:330] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4: stopping tablet replica
I20260811 12:51:23.843538 5115 raft_consensus.cc:2229] T 57287328975f4ecf835d01fc0cc143f2 P ddda3c5ab68642a98b3961e254dc675d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:51:23.843964 5114 raft_consensus.cc:2229] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:51:23.844182 5115 raft_consensus.cc:2258] T 57287328975f4ecf835d01fc0cc143f2 P ddda3c5ab68642a98b3961e254dc675d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:51:23.844508 5116 raft_consensus.cc:2229] T 57287328975f4ecf835d01fc0cc143f2 P f98aa606fbbc4fef924531df98ab46ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:51:23.844700 5114 raft_consensus.cc:2258] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:51:23.845145 5116 raft_consensus.cc:2258] T 57287328975f4ecf835d01fc0cc143f2 P f98aa606fbbc4fef924531df98ab46ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:51:23.846807 5114 ts_tablet_manager.cc:1905] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 12:51:23 dist-test-slave-f0tn krb5kdc[4173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.65@KRBTEST.COM
I20260811 12:51:23.849874 5115 ts_tablet_manager.cc:1905] T 57287328975f4ecf835d01fc0cc143f2 P ddda3c5ab68642a98b3961e254dc675d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:51:23.848667 5116 ts_tablet_manager.cc:1905] T 57287328975f4ecf835d01fc0cc143f2 P f98aa606fbbc4fef924531df98ab46ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:51:23.854688 5114 ts_tablet_manager.cc:1918] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:51:23.854925 5114 log.cc:1196] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/57287328975f4ecf835d01fc0cc143f2
I20260811 12:51:23.855535 5114 ts_tablet_manager.cc:1939] T 57287328975f4ecf835d01fc0cc143f2 P 85aa12e8ca264e318539cd3eca1ebae4: Deleting consensus metadata
I20260811 12:51:23.857455 4221 catalog_manager.cc:4841] TS 85aa12e8ca264e318539cd3eca1ebae4 (127.7.111.66:46107): tablet 57287328975f4ecf835d01fc0cc143f2 (table test-table [id=d161fa40ff7d4951904737be4cd8c71d]) successfully deleted
I20260811 12:51:23.858074 5073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:23.828994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35322 (local address 127.7.111.65:33041)
0811 12:51:23.829206 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:23.829214 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:23.831281 (+ 2067us) server_negotiation.cc:407] Connection header received
0811 12:51:23.831377 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:23.831384 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:23.831505 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:23.831616 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:23.835351 (+ 3735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.836731 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.841606 (+ 4875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:23.842464 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:23.842585 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:23.850042 (+ 7457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:23.850059 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:23.850077 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:23.850124 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:23.854122 (+ 3998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:23.855083 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:23.855093 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:23.855097 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:23.855285 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:23.855791 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:23.855797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:23.855801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:23.856301 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:23.856619 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:23.857469 (+ 850us) server_negotiation.cc:299] Negotiation successful
0811 12:51:23.857703 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:51:23.859097 4582 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:23.863922 5115 ts_tablet_manager.cc:1918] T 57287328975f4ecf835d01fc0cc143f2 P ddda3c5ab68642a98b3961e254dc675d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:51:23.864104 5115 log.cc:1196] T 57287328975f4ecf835d01fc0cc143f2 P ddda3c5ab68642a98b3961e254dc675d: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/57287328975f4ecf835d01fc0cc143f2
I20260811 12:51:23.864711 5115 ts_tablet_manager.cc:1939] T 57287328975f4ecf835d01fc0cc143f2 P ddda3c5ab68642a98b3961e254dc675d: Deleting consensus metadata
I20260811 12:51:23.866806 4220 catalog_manager.cc:4841] TS ddda3c5ab68642a98b3961e254dc675d (127.7.111.67:41629): tablet 57287328975f4ecf835d01fc0cc143f2 (table test-table [id=d161fa40ff7d4951904737be4cd8c71d]) successfully deleted
I20260811 12:51:23.953224 5116 ts_tablet_manager.cc:1918] T 57287328975f4ecf835d01fc0cc143f2 P f98aa606fbbc4fef924531df98ab46ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:51:23.953454 5116 log.cc:1196] T 57287328975f4ecf835d01fc0cc143f2 P f98aa606fbbc4fef924531df98ab46ee: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/57287328975f4ecf835d01fc0cc143f2
I20260811 12:51:23.954061 5116 ts_tablet_manager.cc:1939] T 57287328975f4ecf835d01fc0cc143f2 P f98aa606fbbc4fef924531df98ab46ee: Deleting consensus metadata
I20260811 12:51:23.956338 4224 catalog_manager.cc:4841] TS f98aa606fbbc4fef924531df98ab46ee (127.7.111.65:33041): tablet 57287328975f4ecf835d01fc0cc143f2 (table test-table [id=d161fa40ff7d4951904737be4cd8c71d]) successfully deleted
I20260811 12:51:23.956676 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 4356
I20260811 12:51:23.994490 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 4597
Aug 11 12:51:24 dist-test-slave-f0tn krb5kdc[4173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.66@KRBTEST.COM
I20260811 12:51:24.016506 5066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:23.995755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45672 (local address 127.7.111.66:46107)
0811 12:51:23.995946 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:23.995952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:23.995972 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:51:23.996035 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:23.996040 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:23.996133 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:23.996235 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:24.000013 (+ 3778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:24.001255 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:24.003315 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:24.004033 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:24.004169 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:24.009335 (+ 5166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:24.009349 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:24.009364 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:24.009470 (+ 106us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:24.013194 (+ 3724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:24.014087 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:24.014097 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:24.014101 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:24.014299 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:24.014826 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:24.014832 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:24.014835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:24.015319 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:24.015672 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:24.015847 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 12:51:24.016081 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 12:51:24.017297 4799 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:24.135767 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 4597
I20260811 12:51:24.160836 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 4821
Aug 11 12:51:24 dist-test-slave-f0tn krb5kdc[4173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.67@KRBTEST.COM
I20260811 12:51:24.175661 5075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:24.161729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47926 (local address 127.7.111.67:41629)
0811 12:51:24.161962 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:24.161967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:24.161988 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:51:24.162149 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:24.162153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:24.162232 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:24.162315 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:24.163694 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:24.164488 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:24.165762 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:24.166177 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:24.166284 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:24.170546 (+ 4262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:24.170562 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:24.170581 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:24.170619 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:24.173243 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:24.173856 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:24.173865 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:24.173867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:24.174036 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:24.174339 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:24.174342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:24.174344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:24.174709 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:24.174932 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:24.175128 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 12:51:24.175292 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 12:51:24.176316 5036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:24.264498 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 4821
I20260811 12:51:24.291231 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 4195
I20260811 12:51:24.292322 4296 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:24.402114 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 4195
2026-08-11T12:51:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5246 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T12:51:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:51:24Z Disabled control of system clock
I20260811 12:51:24.500974 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:37927
--webserver_interface=127.7.111.126
--webserver_port=0
--builtin_ntp_servers=127.7.111.84:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:37927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:51:24.839185 5137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:24.839466 5137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:24.839553 5137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:24.847024 5137 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:51:24.847137 5137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:24.847174 5137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:24.847229 5137 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:51:24.847271 5137 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:51:24.856554 5137 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:37927
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:37927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:24.857846 5137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:24.859658 5137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:24.870756 5148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:24.870891 5147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:24.870860 5150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:24.871676 5137 server_base.cc:1034] running on GCE node
I20260811 12:51:24.872430 5137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:24.873765 5137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:24.874970 5137 hybrid_clock.cc:648] HybridClock initialized: now 1786452684874918 us; error 35 us; skew 500 ppm
I20260811 12:51:24.875308 5137 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:51:24.877452 5137 webserver.cc:460] Webserver started at http://127.7.111.126:33157/ using document root <none> and password file <none>
I20260811 12:51:24.878082 5137 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:24.878181 5137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:24.878494 5137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:24.881317 5137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "1e90056ceae64636b99c26434c571028"
format_stamp: "Formatted at 2026-08-11 12:51:24 on dist-test-slave-f0tn"
server_key: "6c875b39a1a556de224f7123f77174da"
server_key_iv: "db42cd0fddeab897578039e8dd544136"
server_key_version: "encryptionkey@0"
I20260811 12:51:24.882144 5137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "1e90056ceae64636b99c26434c571028"
format_stamp: "Formatted at 2026-08-11 12:51:24 on dist-test-slave-f0tn"
server_key: "6c875b39a1a556de224f7123f77174da"
server_key_iv: "db42cd0fddeab897578039e8dd544136"
server_key_version: "encryptionkey@0"
I20260811 12:51:24.887010 5137 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:51:24.890287 5157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:24.891548 5137 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:51:24.891698 5137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "1e90056ceae64636b99c26434c571028"
format_stamp: "Formatted at 2026-08-11 12:51:24 on dist-test-slave-f0tn"
server_key: "6c875b39a1a556de224f7123f77174da"
server_key_iv: "db42cd0fddeab897578039e8dd544136"
server_key_version: "encryptionkey@0"
I20260811 12:51:24.891911 5137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:24.923254 5137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:24.924206 5137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:24.924515 5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:24.942746 5137 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:37927
I20260811 12:51:24.942765 5242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:37927 every 8 connection(s)
I20260811 12:51:24.944186 5137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:51:24.947885 5245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:24.951859 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 5137
I20260811 12:51:24.952029 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:51:24.952520 7613 external_mini_cluster.cc:1428] Setting key 46ad71138b8f7cf408655b09dd5b5ef0
I20260811 12:51:24.959764 5245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028: Bootstrap starting.
I20260811 12:51:24.963291 5245 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:24.964582 5245 log.cc:824] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:24.967909 5245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028: No bootstrap required, opened a new log
I20260811 12:51:24.967916 5249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:24.954427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47090 (local address 127.7.111.126:37927)
0811 12:51:24.955591 (+ 1164us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:24.955605 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:24.955660 (+ 55us) server_negotiation.cc:407] Connection header received
0811 12:51:24.958083 (+ 2423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:24.958111 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:24.958662 (+ 551us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:24.959091 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:24.959656 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:24.961634 (+ 1978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:24.963014 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:24.964066 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:24.964272 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:24.965321 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:24.965329 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:24.965344 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:24.965937 (+ 593us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:24.966033 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:24.966060 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:24.966183 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:24.966381 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 12:51:24.966689 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":361,"threads_started":1}
I20260811 12:51:24.976220 5245 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e90056ceae64636b99c26434c571028" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37927 } }
I20260811 12:51:24.976543 5245 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:24.976614 5245 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e90056ceae64636b99c26434c571028, State: Initialized, Role: FOLLOWER
I20260811 12:51:24.977098 5245 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e90056ceae64636b99c26434c571028" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37927 } }
I20260811 12:51:24.977314 5245 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:51:24.977480 5245 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:51:24.977627 5245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:24.980278 5245 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e90056ceae64636b99c26434c571028" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37927 } }
I20260811 12:51:24.980762 5245 leader_election.cc:302] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e90056ceae64636b99c26434c571028; no voters:
I20260811 12:51:24.981263 5245 leader_election.cc:288] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:51:24.981416 5251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:51:24.982149 5251 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028 [term 1 LEADER]: Becoming Leader. State: Replica: 1e90056ceae64636b99c26434c571028, State: Running, Role: LEADER
I20260811 12:51:24.982645 5251 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e90056ceae64636b99c26434c571028" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37927 } }
I20260811 12:51:24.983361 5245 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:51:24.985491 5254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e90056ceae64636b99c26434c571028" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e90056ceae64636b99c26434c571028" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37927 } } }
I20260811 12:51:24.985466 5256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e90056ceae64636b99c26434c571028. Latest consensus state: current_term: 1 leader_uuid: "1e90056ceae64636b99c26434c571028" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e90056ceae64636b99c26434c571028" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 37927 } } }
I20260811 12:51:24.986052 5254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028 [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:24.986063 5256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028 [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:24.990257 5267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:51:24.995508 5267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:51:25.005831 5267 catalog_manager.cc:1347] Generated new cluster ID: 64e406c376084e879c326d29f1a6c897
I20260811 12:51:25.005964 5267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:51:25.043299 5267 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:51:25.044620 5267 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:51:25.052774 5267 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028: Generated new TSK 0
I20260811 12:51:25.053607 5267 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:51:25.064389 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:37927
--builtin_ntp_servers=127.7.111.84:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:51:25.411252 5285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:25.411533 5285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:25.411629 5285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:25.418679 5285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:25.418774 5285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:25.418884 5285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:51:25.427850 5285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:37927
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:25.429201 5285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:25.431043 5285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:25.443595 5295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:25.443734 5291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:25.443626 5299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:25.444401 5285 server_base.cc:1034] running on GCE node
I20260811 12:51:25.445010 5285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:25.445963 5285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:25.447170 5285 hybrid_clock.cc:648] HybridClock initialized: now 1786452685447109 us; error 56 us; skew 500 ppm
I20260811 12:51:25.447481 5285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:51:25.449651 5285 webserver.cc:460] Webserver started at http://127.7.111.65:46441/ using document root <none> and password file <none>
I20260811 12:51:25.450313 5285 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:25.450417 5285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:25.450726 5285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:25.453531 5285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "dd449c402a034ed5bd3f060ff6c92899"
format_stamp: "Formatted at 2026-08-11 12:51:25 on dist-test-slave-f0tn"
server_key: "948f93e28bf1f96e2432a860d4345ce7"
server_key_iv: "69e1393820395cb48da0dfe6e7626bd5"
server_key_version: "encryptionkey@0"
I20260811 12:51:25.454283 5285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "dd449c402a034ed5bd3f060ff6c92899"
format_stamp: "Formatted at 2026-08-11 12:51:25 on dist-test-slave-f0tn"
server_key: "948f93e28bf1f96e2432a860d4345ce7"
server_key_iv: "69e1393820395cb48da0dfe6e7626bd5"
server_key_version: "encryptionkey@0"
I20260811 12:51:25.459102 5285 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:51:25.462479 5308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:25.463737 5285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:51:25.463891 5285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "dd449c402a034ed5bd3f060ff6c92899"
format_stamp: "Formatted at 2026-08-11 12:51:25 on dist-test-slave-f0tn"
server_key: "948f93e28bf1f96e2432a860d4345ce7"
server_key_iv: "69e1393820395cb48da0dfe6e7626bd5"
server_key_version: "encryptionkey@0"
I20260811 12:51:25.464102 5285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:25.490083 5285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:25.490983 5285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:25.491279 5285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:25.492268 5285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:51:25.493742 5285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:51:25.493841 5285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:25.493945 5285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:51:25.494019 5285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:25.524999 5285 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:41927
I20260811 12:51:25.525065 5491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:41927 every 8 connection(s)
I20260811 12:51:25.526312 5285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:51:25.531526 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 5285
I20260811 12:51:25.531689 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:51:25.532249 7613 external_mini_cluster.cc:1428] Setting key bea5b9c8a1dbd3440e18824afe1e76cd
I20260811 12:51:25.535247 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:37927
--builtin_ntp_servers=127.7.111.84:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:51:25.539301 5499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:25.528771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:50529 (local address 127.7.111.126:37927)
0811 12:51:25.529180 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:25.529184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:25.530307 (+ 1123us) server_negotiation.cc:407] Connection header received
0811 12:51:25.532458 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:25.532463 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:25.532536 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:25.532630 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:25.533840 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:25.534627 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:25.536704 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:25.537097 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:25.537199 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:25.538605 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:25.538609 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:25.538612 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:25.538630 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:25.538680 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:25.538692 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:25.538758 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:25.538967 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 12:51:25.539041 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":204,"threads_started":1}
I20260811 12:51:25.540480 5502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:25.529246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:37927 (local address 127.7.111.65:50529)
0811 12:51:25.530145 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:25.530180 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:25.532149 (+ 1969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:25.532876 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:25.532884 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:25.532974 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:51:25.533681 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:25.533704 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:25.534782 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:25.534787 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:25.536569 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:25.536584 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:25.537320 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:25.537327 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:25.537628 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:25.538293 (+ 665us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:51:25.538296 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:51:25.538343 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:51:25.538433 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:51:25.538778 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:51:25.538787 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:51:25.538795 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:51:25.538950 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 12:51:25.539312 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":260,"threads_started":1}
I20260811 12:51:25.541905 5492 heartbeater.cc:347] Connected to a master server at 127.7.111.126:37927
I20260811 12:51:25.542198 5492 heartbeater.cc:464] Registering TS with master...
I20260811 12:51:25.543025 5492 heartbeater.cc:511] Master 127.7.111.126:37927 requested a full tablet report, sending...
I20260811 12:51:25.545056 5182 ts_manager.cc:194] Registered new tserver with Master: dd449c402a034ed5bd3f060ff6c92899 (127.7.111.65:41927)
I20260811 12:51:25.546545 5182 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.111.65:50529
W20260811 12:51:25.869017 5503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:25.869297 5503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:25.869418 5503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:25.876516 5503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:25.876614 5503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:25.876724 5503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:51:25.885865 5503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:37927
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:25.887125 5503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:25.888881 5503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:25.901621 5517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:25.901574 5514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:25.901600 5519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:25.902395 5503 server_base.cc:1034] running on GCE node
I20260811 12:51:25.902958 5503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:25.903901 5503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:25.905102 5503 hybrid_clock.cc:648] HybridClock initialized: now 1786452685905039 us; error 53 us; skew 500 ppm
I20260811 12:51:25.905448 5503 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:51:25.907696 5503 webserver.cc:460] Webserver started at http://127.7.111.66:41057/ using document root <none> and password file <none>
I20260811 12:51:25.908406 5503 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:25.908504 5503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:25.908835 5503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:25.911844 5503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "3bdff9352d6e460ab04a1a74b85e4c5a"
format_stamp: "Formatted at 2026-08-11 12:51:25 on dist-test-slave-f0tn"
server_key: "b78ab6392cff0b40b560382ea945c5be"
server_key_iv: "38eed475c1e46fdb5324a43741b58c62"
server_key_version: "encryptionkey@0"
I20260811 12:51:25.912614 5503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "3bdff9352d6e460ab04a1a74b85e4c5a"
format_stamp: "Formatted at 2026-08-11 12:51:25 on dist-test-slave-f0tn"
server_key: "b78ab6392cff0b40b560382ea945c5be"
server_key_iv: "38eed475c1e46fdb5324a43741b58c62"
server_key_version: "encryptionkey@0"
I20260811 12:51:25.917613 5503 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:51:25.920905 5530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:25.922206 5503 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:51:25.922348 5503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "3bdff9352d6e460ab04a1a74b85e4c5a"
format_stamp: "Formatted at 2026-08-11 12:51:25 on dist-test-slave-f0tn"
server_key: "b78ab6392cff0b40b560382ea945c5be"
server_key_iv: "38eed475c1e46fdb5324a43741b58c62"
server_key_version: "encryptionkey@0"
I20260811 12:51:25.922547 5503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:25.943354 5503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:25.944149 5503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:25.944442 5503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:25.945513 5503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:51:25.947011 5503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:51:25.947117 5503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:25.947222 5503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:51:25.947293 5503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:25.979157 5503 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:34839
I20260811 12:51:25.979228 5699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:34839 every 8 connection(s)
I20260811 12:51:25.980525 5503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:51:25.986586 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 5503
I20260811 12:51:25.986793 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:51:25.987254 7613 external_mini_cluster.cc:1428] Setting key 9da09c1306d5216a9f4a1204836fef94
I20260811 12:51:25.990181 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:37927
--builtin_ntp_servers=127.7.111.84:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:51:25.993157 5499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:25.983163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:51659 (local address 127.7.111.126:37927)
0811 12:51:25.983370 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:25.983377 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:25.984743 (+ 1366us) server_negotiation.cc:407] Connection header received
0811 12:51:25.986662 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:25.986669 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:25.986774 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:25.986894 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:25.988066 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:25.988816 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:25.990524 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:25.991073 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:25.991195 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:25.992457 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:25.992463 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:25.992467 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:25.992488 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:25.992524 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:25.992538 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:25.992620 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:25.992793 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 12:51:25.992889 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:51:25.994186 5704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:25.983694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:37927 (local address 127.7.111.66:51659)
0811 12:51:25.984581 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:25.984623 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:25.986350 (+ 1727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:25.987158 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:25.987166 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:25.987273 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:51:25.987889 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:25.987909 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:25.988963 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:25.988967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:25.990365 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:25.990381 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:25.991286 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:25.991291 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:25.991494 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:25.992140 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:51:25.992143 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:51:25.992193 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:51:25.992277 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:51:25.992620 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:51:25.992626 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:51:25.992631 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 12:51:25.992765 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 12:51:25.993061 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":255,"threads_started":1}
I20260811 12:51:25.995551 5701 heartbeater.cc:347] Connected to a master server at 127.7.111.126:37927
I20260811 12:51:25.995855 5701 heartbeater.cc:464] Registering TS with master...
I20260811 12:51:25.996670 5701 heartbeater.cc:511] Master 127.7.111.126:37927 requested a full tablet report, sending...
I20260811 12:51:25.998049 5182 ts_manager.cc:194] Registered new tserver with Master: 3bdff9352d6e460ab04a1a74b85e4c5a (127.7.111.66:34839)
I20260811 12:51:25.998852 5182 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.111.66:51659
W20260811 12:51:26.332130 5705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:26.332408 5705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:26.332497 5705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:26.339502 5705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:26.339598 5705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:26.339709 5705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:51:26.348718 5705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:37927
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:26.350024 5705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:26.351790 5705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:26.364591 5712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:26.364523 5713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:26.364564 5722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:26.365486 5705 server_base.cc:1034] running on GCE node
I20260811 12:51:26.366048 5705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:26.366902 5705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:26.368104 5705 hybrid_clock.cc:648] HybridClock initialized: now 1786452686368057 us; error 39 us; skew 500 ppm
I20260811 12:51:26.368445 5705 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:51:26.370631 5705 webserver.cc:460] Webserver started at http://127.7.111.67:42755/ using document root <none> and password file <none>
I20260811 12:51:26.371294 5705 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:26.371397 5705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:26.371719 5705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:26.374822 5705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "d4cd8bce735a4d5eb96aefa9a3da4797"
format_stamp: "Formatted at 2026-08-11 12:51:26 on dist-test-slave-f0tn"
server_key: "3dafbaeb24eae7f7feffcf62ca77e9a8"
server_key_iv: "d268fdea08fb7b3de95c5d67f0945b3d"
server_key_version: "encryptionkey@0"
I20260811 12:51:26.375624 5705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "d4cd8bce735a4d5eb96aefa9a3da4797"
format_stamp: "Formatted at 2026-08-11 12:51:26 on dist-test-slave-f0tn"
server_key: "3dafbaeb24eae7f7feffcf62ca77e9a8"
server_key_iv: "d268fdea08fb7b3de95c5d67f0945b3d"
server_key_version: "encryptionkey@0"
I20260811 12:51:26.380632 5705 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:51:26.384145 5730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:26.385452 5705 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:51:26.385602 5705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "d4cd8bce735a4d5eb96aefa9a3da4797"
format_stamp: "Formatted at 2026-08-11 12:51:26 on dist-test-slave-f0tn"
server_key: "3dafbaeb24eae7f7feffcf62ca77e9a8"
server_key_iv: "d268fdea08fb7b3de95c5d67f0945b3d"
server_key_version: "encryptionkey@0"
I20260811 12:51:26.385809 5705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:26.398711 5705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:26.399451 5705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:26.399727 5705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:26.400640 5705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:51:26.402100 5705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:51:26.402196 5705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:26.402304 5705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:51:26.402374 5705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:26.432557 5705 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:46289
I20260811 12:51:26.432603 5908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:46289 every 8 connection(s)
I20260811 12:51:26.433856 5705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:51:26.443293 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 5705
I20260811 12:51:26.443439 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:51:26.443871 7613 external_mini_cluster.cc:1428] Setting key 178590c10ec0cdddd4d5e548e05dc382
I20260811 12:51:26.446280 5499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:26.436340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:60863 (local address 127.7.111.126:37927)
0811 12:51:26.436508 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:26.436513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:26.437921 (+ 1408us) server_negotiation.cc:407] Connection header received
0811 12:51:26.439662 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:26.439667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:26.439761 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:26.439849 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:26.440891 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.442016 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.443559 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.443968 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.444088 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:26.445473 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:26.445479 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:26.445482 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:26.445500 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:26.445535 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:26.445548 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:26.445624 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:26.445845 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 12:51:26.445939 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:51:26.447230 5911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:26.436859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:37927 (local address 127.7.111.67:60863)
0811 12:51:26.437763 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:26.437798 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:26.439402 (+ 1604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:26.440085 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:26.440093 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:26.440174 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:51:26.440728 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:26.440744 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.442157 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.442161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:26.443402 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:26.443418 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.444171 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.444178 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:26.444477 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:26.445144 (+ 667us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:51:26.445147 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:51:26.445186 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:51:26.445259 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:51:26.445639 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:51:26.445651 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:51:26.445660 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 12:51:26.445819 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 12:51:26.446182 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":286,"threads_started":1}
I20260811 12:51:26.448405 5909 heartbeater.cc:347] Connected to a master server at 127.7.111.126:37927
I20260811 12:51:26.448666 5909 heartbeater.cc:464] Registering TS with master...
I20260811 12:51:26.449363 5909 heartbeater.cc:511] Master 127.7.111.126:37927 requested a full tablet report, sending...
I20260811 12:51:26.450641 5182 ts_manager.cc:194] Registered new tserver with Master: d4cd8bce735a4d5eb96aefa9a3da4797 (127.7.111.67:46289)
I20260811 12:51:26.451365 5182 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.111.67:60863
I20260811 12:51:26.458405 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:51:26.467629 5499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:26.462494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47104 (local address 127.7.111.126:37927)
0811 12:51:26.462670 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:26.462674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:26.463056 (+ 382us) server_negotiation.cc:407] Connection header received
0811 12:51:26.463184 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:26.463188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:26.463254 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:26.463330 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:26.463745 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.464418 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.465754 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.466148 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.466245 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:26.467118 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:26.467122 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:26.467125 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:26.467140 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:26.467163 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:26.467184 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:26.467243 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:26.467329 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 12:51:26.467391 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 12:51:26.471175 5182 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47104:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:51:26.477859 5182 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:51:26.502091 5930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:26.489920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:41927 (local address 127.0.0.1:43408)
0811 12:51:26.492043 (+ 2123us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:26.492061 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:26.492187 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:26.493202 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:26.493210 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:26.493244 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:26.493671 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:26.493693 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.495854 (+ 2161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.495860 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:26.498524 (+ 2664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:26.498542 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.500985 (+ 2443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.500993 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:26.501237 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:26.501293 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 12:51:26.501451 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 12:51:26.501687 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1964,"thread_start_us":1818,"threads_started":1}
I20260811 12:51:26.503109 5935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:26.490526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43408 (local address 127.7.111.65:41927)
0811 12:51:26.491580 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:26.491590 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:26.492317 (+ 727us) server_negotiation.cc:407] Connection header received
0811 12:51:26.492437 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:26.492446 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:26.492731 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:26.492926 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:26.493918 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.495663 (+ 1745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.498794 (+ 3131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.500391 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.500535 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:26.500833 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:26.502519 (+ 1686us) server_negotiation.cc:299] Negotiation successful
0811 12:51:26.502746 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":866,"thread_start_us":753,"threads_started":1}
I20260811 12:51:26.510062 5382 tablet_service.cc:1467] Processing CreateTablet for tablet 18b436754fc645bb92728cc4ad61176f (DEFAULT_TABLE table=test-table [id=f548c9297c084063b025a978f6673d87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:51:26.511089 5927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:26.489559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:46289 (local address 127.0.0.1:54666)
0811 12:51:26.490725 (+ 1166us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:26.490768 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:26.490965 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:26.505107 (+ 14142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:26.505112 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:26.505139 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:26.505547 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:26.505560 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.507502 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.507507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:26.508900 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:26.508916 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.510326 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.510333 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:26.510498 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:26.510530 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:51:26.510620 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 12:51:26.510751 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"mutex_wait_us":1,"thread_start_us":662,"threads_started":1}
I20260811 12:51:26.511189 5932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:26.489901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54666 (local address 127.7.111.67:46289)
0811 12:51:26.504572 (+ 14671us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:26.504578 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:26.504608 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:51:26.504708 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:26.504714 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:26.504913 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:26.505097 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:26.505789 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.507346 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.509088 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.510118 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.510240 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:26.510531 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:26.510683 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 12:51:26.510842 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14512,"thread_start_us":14348,"threads_started":1}
I20260811 12:51:26.512806 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:26.492468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47596 (local address 127.7.111.66:34839)
0811 12:51:26.505787 (+ 13319us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:26.505795 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:26.505822 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:51:26.505916 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:26.505924 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:26.506156 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:26.506355 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:26.506781 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.508201 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.510419 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.511778 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.511911 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:26.512137 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:26.512322 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 12:51:26.512481 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13142,"thread_start_us":13028,"threads_started":1}
I20260811 12:51:26.512699 5930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:26.492078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:34839 (local address 127.0.0.1:47596)
0811 12:51:26.503832 (+ 11754us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:26.503853 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:26.503982 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:26.506356 (+ 2374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:26.506361 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:26.506387 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:26.506595 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:26.506606 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.508369 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.508374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:26.510225 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:26.510240 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.511948 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.511954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:26.512132 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:26.512165 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:51:26.512272 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 12:51:26.512406 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11549,"thread_start_us":13776,"threads_started":1}
I20260811 12:51:26.519176 5599 tablet_service.cc:1467] Processing CreateTablet for tablet 18b436754fc645bb92728cc4ad61176f (DEFAULT_TABLE table=test-table [id=f548c9297c084063b025a978f6673d87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:51:26.520398 5807 tablet_service.cc:1467] Processing CreateTablet for tablet 18b436754fc645bb92728cc4ad61176f (DEFAULT_TABLE table=test-table [id=f548c9297c084063b025a978f6673d87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:51:26.521597 5382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18b436754fc645bb92728cc4ad61176f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:26.521924 5807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18b436754fc645bb92728cc4ad61176f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:26.526083 5599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18b436754fc645bb92728cc4ad61176f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:26.544665 5943 tablet_bootstrap.cc:492] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797: Bootstrap starting.
I20260811 12:51:26.547500 5945 tablet_bootstrap.cc:492] T 18b436754fc645bb92728cc4ad61176f P 3bdff9352d6e460ab04a1a74b85e4c5a: Bootstrap starting.
I20260811 12:51:26.549005 5943 tablet_bootstrap.cc:654] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:26.549726 5492 heartbeater.cc:503] Master 127.7.111.126:37927 was elected leader, sending a full tablet report...
I20260811 12:51:26.550451 5943 log.cc:824] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:26.550547 5945 tablet_bootstrap.cc:654] T 18b436754fc645bb92728cc4ad61176f P 3bdff9352d6e460ab04a1a74b85e4c5a: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:26.551740 5945 log.cc:824] T 18b436754fc645bb92728cc4ad61176f P 3bdff9352d6e460ab04a1a74b85e4c5a: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:26.553836 5946 tablet_bootstrap.cc:492] T 18b436754fc645bb92728cc4ad61176f P dd449c402a034ed5bd3f060ff6c92899: Bootstrap starting.
I20260811 12:51:26.556655 5943 tablet_bootstrap.cc:492] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797: No bootstrap required, opened a new log
I20260811 12:51:26.556819 5946 tablet_bootstrap.cc:654] T 18b436754fc645bb92728cc4ad61176f P dd449c402a034ed5bd3f060ff6c92899: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:26.556952 5943 ts_tablet_manager.cc:1397] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.004s
I20260811 12:51:26.557019 5945 tablet_bootstrap.cc:492] T 18b436754fc645bb92728cc4ad61176f P 3bdff9352d6e460ab04a1a74b85e4c5a: No bootstrap required, opened a new log
I20260811 12:51:26.557268 5945 ts_tablet_manager.cc:1397] T 18b436754fc645bb92728cc4ad61176f P 3bdff9352d6e460ab04a1a74b85e4c5a: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260811 12:51:26.557844 5946 log.cc:824] T 18b436754fc645bb92728cc4ad61176f P dd449c402a034ed5bd3f060ff6c92899: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:26.560963 5946 tablet_bootstrap.cc:492] T 18b436754fc645bb92728cc4ad61176f P dd449c402a034ed5bd3f060ff6c92899: No bootstrap required, opened a new log
I20260811 12:51:26.561203 5946 ts_tablet_manager.cc:1397] T 18b436754fc645bb92728cc4ad61176f P dd449c402a034ed5bd3f060ff6c92899: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 12:51:26.567453 5943 raft_consensus.cc:348] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cd8bce735a4d5eb96aefa9a3da4797" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 46289 } } peers { permanent_uuid: "dd449c402a034ed5bd3f060ff6c92899" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41927 } } peers { permanent_uuid: "3bdff9352d6e460ab04a1a74b85e4c5a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 34839 } }
I20260811 12:51:26.567453 5945 raft_consensus.cc:348] T 18b436754fc645bb92728cc4ad61176f P 3bdff9352d6e460ab04a1a74b85e4c5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cd8bce735a4d5eb96aefa9a3da4797" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 46289 } } peers { permanent_uuid: "dd449c402a034ed5bd3f060ff6c92899" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41927 } } peers { permanent_uuid: "3bdff9352d6e460ab04a1a74b85e4c5a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 34839 } }
I20260811 12:51:26.567875 5943 raft_consensus.cc:374] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:26.567875 5945 raft_consensus.cc:374] T 18b436754fc645bb92728cc4ad61176f P 3bdff9352d6e460ab04a1a74b85e4c5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:26.567967 5945 raft_consensus.cc:729] T 18b436754fc645bb92728cc4ad61176f P 3bdff9352d6e460ab04a1a74b85e4c5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bdff9352d6e460ab04a1a74b85e4c5a, State: Initialized, Role: FOLLOWER
I20260811 12:51:26.567976 5943 raft_consensus.cc:729] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4cd8bce735a4d5eb96aefa9a3da4797, State: Initialized, Role: FOLLOWER
I20260811 12:51:26.567960 5946 raft_consensus.cc:348] T 18b436754fc645bb92728cc4ad61176f P dd449c402a034ed5bd3f060ff6c92899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cd8bce735a4d5eb96aefa9a3da4797" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 46289 } } peers { permanent_uuid: "dd449c402a034ed5bd3f060ff6c92899" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41927 } } peers { permanent_uuid: "3bdff9352d6e460ab04a1a74b85e4c5a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 34839 } }
I20260811 12:51:26.568390 5946 raft_consensus.cc:374] T 18b436754fc645bb92728cc4ad61176f P dd449c402a034ed5bd3f060ff6c92899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:26.568490 5946 raft_consensus.cc:729] T 18b436754fc645bb92728cc4ad61176f P dd449c402a034ed5bd3f060ff6c92899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd449c402a034ed5bd3f060ff6c92899, State: Initialized, Role: FOLLOWER
I20260811 12:51:26.568571 5943 consensus_queue.cc:260] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cd8bce735a4d5eb96aefa9a3da4797" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 46289 } } peers { permanent_uuid: "dd449c402a034ed5bd3f060ff6c92899" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41927 } } peers { permanent_uuid: "3bdff9352d6e460ab04a1a74b85e4c5a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 34839 } }
I20260811 12:51:26.568682 5945 consensus_queue.cc:260] T 18b436754fc645bb92728cc4ad61176f P 3bdff9352d6e460ab04a1a74b85e4c5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cd8bce735a4d5eb96aefa9a3da4797" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 46289 } } peers { permanent_uuid: "dd449c402a034ed5bd3f060ff6c92899" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41927 } } peers { permanent_uuid: "3bdff9352d6e460ab04a1a74b85e4c5a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 34839 } }
I20260811 12:51:26.569029 5946 consensus_queue.cc:260] T 18b436754fc645bb92728cc4ad61176f P dd449c402a034ed5bd3f060ff6c92899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cd8bce735a4d5eb96aefa9a3da4797" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 46289 } } peers { permanent_uuid: "dd449c402a034ed5bd3f060ff6c92899" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41927 } } peers { permanent_uuid: "3bdff9352d6e460ab04a1a74b85e4c5a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 34839 } }
I20260811 12:51:26.569829 5909 heartbeater.cc:503] Master 127.7.111.126:37927 was elected leader, sending a full tablet report...
I20260811 12:51:26.570425 5943 ts_tablet_manager.cc:1428] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260811 12:51:26.572180 5946 ts_tablet_manager.cc:1428] T 18b436754fc645bb92728cc4ad61176f P dd449c402a034ed5bd3f060ff6c92899: Time spent starting tablet: real 0.011s user 0.009s sys 0.000s
I20260811 12:51:26.572966 5701 heartbeater.cc:503] Master 127.7.111.126:37927 was elected leader, sending a full tablet report...
I20260811 12:51:26.573686 5945 ts_tablet_manager.cc:1428] T 18b436754fc645bb92728cc4ad61176f P 3bdff9352d6e460ab04a1a74b85e4c5a: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
W20260811 12:51:26.685519 5910 tablet.cc:2369] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:51:26.711982 5952 raft_consensus.cc:482] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:51:26.712277 5952 raft_consensus.cc:504] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cd8bce735a4d5eb96aefa9a3da4797" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 46289 } } peers { permanent_uuid: "dd449c402a034ed5bd3f060ff6c92899" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41927 } } peers { permanent_uuid: "3bdff9352d6e460ab04a1a74b85e4c5a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 34839 } }
I20260811 12:51:26.714028 5952 leader_election.cc:288] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd449c402a034ed5bd3f060ff6c92899 (127.7.111.65:41927), 3bdff9352d6e460ab04a1a74b85e4c5a (127.7.111.66:34839)
I20260811 12:51:26.720292 5935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:26.714507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:47205 (local address 127.7.111.65:41927)
0811 12:51:26.714714 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:26.714719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:26.714742 (+ 23us) server_negotiation.cc:407] Connection header received
0811 12:51:26.714821 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:26.714826 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:26.714917 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:26.715003 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:26.715621 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.716577 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.718455 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.719523 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.719633 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:26.719681 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:26.720007 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 12:51:26.720090 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:51:26.720654 5911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:26.714342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:41927 (local address 127.7.111.67:47205)
0811 12:51:26.714524 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:26.714541 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:26.714696 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:26.715247 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:26.715251 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:26.715277 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:26.715443 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:26.715450 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.716703 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.716709 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:26.718298 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:26.718311 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.719665 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.719671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:26.719858 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:26.719894 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 12:51:26.719991 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 12:51:26.720130 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 12:51:26.721083 5955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:26.714772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:34839 (local address 127.7.111.67:50047)
0811 12:51:26.715419 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:26.715434 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:26.715547 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:26.715845 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:26.715850 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:26.715875 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:26.716252 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:26.716263 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.717456 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.717463 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:26.719076 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:26.719091 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.720419 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.720425 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:26.720581 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:26.720604 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 12:51:26.720683 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 12:51:26.720805 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":314,"threads_started":1}
I20260811 12:51:26.721112 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:26.714960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:50047 (local address 127.7.111.66:34839)
0811 12:51:26.715136 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:26.715141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:26.715521 (+ 380us) server_negotiation.cc:407] Connection header received
0811 12:51:26.715663 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:26.715666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:26.715742 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:26.715808 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:26.716381 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.717283 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.719265 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.720252 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.720394 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:26.720458 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:26.720743 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 12:51:26.720867 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:51:26.721773 5416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18b436754fc645bb92728cc4ad61176f" candidate_uuid: "d4cd8bce735a4d5eb96aefa9a3da4797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd449c402a034ed5bd3f060ff6c92899" is_pre_election: true
I20260811 12:51:26.722126 5627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18b436754fc645bb92728cc4ad61176f" candidate_uuid: "d4cd8bce735a4d5eb96aefa9a3da4797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdff9352d6e460ab04a1a74b85e4c5a" is_pre_election: true
I20260811 12:51:26.722256 5416 raft_consensus.cc:2454] T 18b436754fc645bb92728cc4ad61176f P dd449c402a034ed5bd3f060ff6c92899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4cd8bce735a4d5eb96aefa9a3da4797 in term 0.
I20260811 12:51:26.722595 5627 raft_consensus.cc:2454] T 18b436754fc645bb92728cc4ad61176f P 3bdff9352d6e460ab04a1a74b85e4c5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4cd8bce735a4d5eb96aefa9a3da4797 in term 0.
I20260811 12:51:26.722911 5733 leader_election.cc:302] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4cd8bce735a4d5eb96aefa9a3da4797, dd449c402a034ed5bd3f060ff6c92899; no voters:
I20260811 12:51:26.723279 5952 raft_consensus.cc:2781] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:51:26.723407 5952 raft_consensus.cc:482] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:51:26.723515 5952 raft_consensus.cc:3037] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:26.725741 5952 raft_consensus.cc:504] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cd8bce735a4d5eb96aefa9a3da4797" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 46289 } } peers { permanent_uuid: "dd449c402a034ed5bd3f060ff6c92899" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41927 } } peers { permanent_uuid: "3bdff9352d6e460ab04a1a74b85e4c5a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 34839 } }
I20260811 12:51:26.726385 5952 leader_election.cc:288] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797 [CANDIDATE]: Term 1 election: Requested vote from peers dd449c402a034ed5bd3f060ff6c92899 (127.7.111.65:41927), 3bdff9352d6e460ab04a1a74b85e4c5a (127.7.111.66:34839)
I20260811 12:51:26.726747 5416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18b436754fc645bb92728cc4ad61176f" candidate_uuid: "d4cd8bce735a4d5eb96aefa9a3da4797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd449c402a034ed5bd3f060ff6c92899"
I20260811 12:51:26.726913 5627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18b436754fc645bb92728cc4ad61176f" candidate_uuid: "d4cd8bce735a4d5eb96aefa9a3da4797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdff9352d6e460ab04a1a74b85e4c5a"
I20260811 12:51:26.726961 5416 raft_consensus.cc:3037] T 18b436754fc645bb92728cc4ad61176f P dd449c402a034ed5bd3f060ff6c92899 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:26.727109 5627 raft_consensus.cc:3037] T 18b436754fc645bb92728cc4ad61176f P 3bdff9352d6e460ab04a1a74b85e4c5a [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:26.730134 5416 raft_consensus.cc:2454] T 18b436754fc645bb92728cc4ad61176f P dd449c402a034ed5bd3f060ff6c92899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4cd8bce735a4d5eb96aefa9a3da4797 in term 1.
I20260811 12:51:26.730144 5627 raft_consensus.cc:2454] T 18b436754fc645bb92728cc4ad61176f P 3bdff9352d6e460ab04a1a74b85e4c5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4cd8bce735a4d5eb96aefa9a3da4797 in term 1.
I20260811 12:51:26.730487 5733 leader_election.cc:302] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4cd8bce735a4d5eb96aefa9a3da4797, dd449c402a034ed5bd3f060ff6c92899; no voters:
I20260811 12:51:26.730751 5952 raft_consensus.cc:2781] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:51:26.731178 5952 raft_consensus.cc:686] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797 [term 1 LEADER]: Becoming Leader. State: Replica: d4cd8bce735a4d5eb96aefa9a3da4797, State: Running, Role: LEADER
I20260811 12:51:26.731611 5952 consensus_queue.cc:237] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cd8bce735a4d5eb96aefa9a3da4797" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 46289 } } peers { permanent_uuid: "dd449c402a034ed5bd3f060ff6c92899" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41927 } } peers { permanent_uuid: "3bdff9352d6e460ab04a1a74b85e4c5a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 34839 } }
W20260811 12:51:26.732506 5703 tablet.cc:2369] T 18b436754fc645bb92728cc4ad61176f P 3bdff9352d6e460ab04a1a74b85e4c5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:51:26.736248 5178 catalog_manager.cc:5495] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4cd8bce735a4d5eb96aefa9a3da4797 (127.7.111.67). New cstate: current_term: 1 leader_uuid: "d4cd8bce735a4d5eb96aefa9a3da4797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cd8bce735a4d5eb96aefa9a3da4797" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 46289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd449c402a034ed5bd3f060ff6c92899" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bdff9352d6e460ab04a1a74b85e4c5a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 34839 } health_report { overall_health: UNKNOWN } } }
W20260811 12:51:26.778640 5495 tablet.cc:2369] T 18b436754fc645bb92728cc4ad61176f P dd449c402a034ed5bd3f060ff6c92899: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:51:26.795917 5932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:26.790564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54680 (local address 127.7.111.67:46289)
0811 12:51:26.790774 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:26.790780 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:26.790796 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:51:26.790883 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:26.790887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:26.790962 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:26.791092 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:51:26.791476 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.792205 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.793692 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.794116 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.794226 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:26.794723 (+ 497us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:51:26.795275 (+ 552us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:51:26.795432 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:26.795552 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 12:51:26.795645 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:51:26.804078 5627 raft_consensus.cc:1261] T 18b436754fc645bb92728cc4ad61176f P 3bdff9352d6e460ab04a1a74b85e4c5a [term 1 FOLLOWER]: Refusing update from remote peer d4cd8bce735a4d5eb96aefa9a3da4797: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:51:26.804075 5416 raft_consensus.cc:1261] T 18b436754fc645bb92728cc4ad61176f P dd449c402a034ed5bd3f060ff6c92899 [term 1 FOLLOWER]: Refusing update from remote peer d4cd8bce735a4d5eb96aefa9a3da4797: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:51:26.805047 5956 consensus_queue.cc:1035] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bdff9352d6e460ab04a1a74b85e4c5a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 34839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:51:26.805418 5952 consensus_queue.cc:1035] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd449c402a034ed5bd3f060ff6c92899" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 41927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:51:26.815539 5962 mvcc.cc:205] Tried to move back new op lower bound from 7317310205138837504 to 7317310204856389632. Current Snapshot: MvccSnapshot[applied={T|T < 7317310205138837504}]
I20260811 12:51:26.816231 5965 mvcc.cc:205] Tried to move back new op lower bound from 7317310205138837504 to 7317310204856389632. Current Snapshot: MvccSnapshot[applied={T|T < 7317310205138837504}]
I20260811 12:51:26.829610 5178 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:51:26.829841 5178 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:51:26.832968 5178 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e90056ceae64636b99c26434c571028: Sending DeleteTablet for 3 replicas of tablet 18b436754fc645bb92728cc4ad61176f
I20260811 12:51:26.833938 5807 tablet_service.cc:1514] Processing DeleteTablet for tablet 18b436754fc645bb92728cc4ad61176f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:51:26 UTC) from {username='slave'} at 127.0.0.1:54666
I20260811 12:51:26.834780 5982 tablet_replica.cc:330] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797: stopping tablet replica
I20260811 12:51:26.834772 5599 tablet_service.cc:1514] Processing DeleteTablet for tablet 18b436754fc645bb92728cc4ad61176f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:51:26 UTC) from {username='slave'} at 127.0.0.1:47596
I20260811 12:51:26.835068 5382 tablet_service.cc:1514] Processing DeleteTablet for tablet 18b436754fc645bb92728cc4ad61176f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:51:26 UTC) from {username='slave'} at 127.0.0.1:43408
I20260811 12:51:26.835305 5982 raft_consensus.cc:2229] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:51:26.835964 5982 raft_consensus.cc:2258] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:51:26.836896 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 5285
I20260811 12:51:26.838394 5982 ts_tablet_manager.cc:1905] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:51:26.845844 5982 ts_tablet_manager.cc:1918] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:51:26.846084 5982 log.cc:1196] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/18b436754fc645bb92728cc4ad61176f
I20260811 12:51:26.846730 5982 ts_tablet_manager.cc:1939] T 18b436754fc645bb92728cc4ad61176f P d4cd8bce735a4d5eb96aefa9a3da4797: Deleting consensus metadata
I20260811 12:51:26.848608 5163 catalog_manager.cc:4841] TS d4cd8bce735a4d5eb96aefa9a3da4797 (127.7.111.67:46289): tablet 18b436754fc645bb92728cc4ad61176f (table test-table [id=f548c9297c084063b025a978f6673d87]) successfully deleted
I20260811 12:51:26.852402 5983 tablet_replica.cc:330] T 18b436754fc645bb92728cc4ad61176f P 3bdff9352d6e460ab04a1a74b85e4c5a: stopping tablet replica
I20260811 12:51:26.852913 5983 raft_consensus.cc:2229] T 18b436754fc645bb92728cc4ad61176f P 3bdff9352d6e460ab04a1a74b85e4c5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:51:26.853559 5983 raft_consensus.cc:2258] T 18b436754fc645bb92728cc4ad61176f P 3bdff9352d6e460ab04a1a74b85e4c5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:51:26.854192 5984 tablet_replica.cc:330] T 18b436754fc645bb92728cc4ad61176f P dd449c402a034ed5bd3f060ff6c92899: stopping tablet replica
I20260811 12:51:26.854580 5984 raft_consensus.cc:2229] T 18b436754fc645bb92728cc4ad61176f P dd449c402a034ed5bd3f060ff6c92899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:51:26.855041 5984 raft_consensus.cc:2258] T 18b436754fc645bb92728cc4ad61176f P dd449c402a034ed5bd3f060ff6c92899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:51:26.857528 5983 ts_tablet_manager.cc:1905] T 18b436754fc645bb92728cc4ad61176f P 3bdff9352d6e460ab04a1a74b85e4c5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:51:26.858599 5984 ts_tablet_manager.cc:1905] T 18b436754fc645bb92728cc4ad61176f P dd449c402a034ed5bd3f060ff6c92899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:51:26.863286 5935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:26.838479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43422 (local address 127.7.111.65:41927)
0811 12:51:26.838660 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:26.838666 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:26.851755 (+ 13089us) server_negotiation.cc:407] Connection header received
0811 12:51:26.851947 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:26.851953 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:26.852091 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:26.852218 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:26.852929 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.854155 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.855552 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:26.856161 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:26.856260 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:26.857477 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:26.857487 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:26.857503 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:26.858248 (+ 745us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:26.858306 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:26.858331 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:26.858445 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:26.862819 (+ 4374us) server_negotiation.cc:299] Negotiation successful
0811 12:51:26.862930 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:51:26.863572 5984 ts_tablet_manager.cc:1918] T 18b436754fc645bb92728cc4ad61176f P dd449c402a034ed5bd3f060ff6c92899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:51:26.863730 5984 log.cc:1196] T 18b436754fc645bb92728cc4ad61176f P dd449c402a034ed5bd3f060ff6c92899: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/18b436754fc645bb92728cc4ad61176f
I20260811 12:51:26.864296 5984 ts_tablet_manager.cc:1939] T 18b436754fc645bb92728cc4ad61176f P dd449c402a034ed5bd3f060ff6c92899: Deleting consensus metadata
I20260811 12:51:26.864403 5983 ts_tablet_manager.cc:1918] T 18b436754fc645bb92728cc4ad61176f P 3bdff9352d6e460ab04a1a74b85e4c5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:51:26.864550 5983 log.cc:1196] T 18b436754fc645bb92728cc4ad61176f P 3bdff9352d6e460ab04a1a74b85e4c5a: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/18b436754fc645bb92728cc4ad61176f
I20260811 12:51:26.864835 5486 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:26.865067 5983 ts_tablet_manager.cc:1939] T 18b436754fc645bb92728cc4ad61176f P 3bdff9352d6e460ab04a1a74b85e4c5a: Deleting consensus metadata
I20260811 12:51:26.866909 5161 catalog_manager.cc:4841] TS 3bdff9352d6e460ab04a1a74b85e4c5a (127.7.111.66:34839): tablet 18b436754fc645bb92728cc4ad61176f (table test-table [id=f548c9297c084063b025a978f6673d87]) successfully deleted
I20260811 12:51:27.001084 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 5285
I20260811 12:51:27.001317 5161 catalog_manager.cc:4841] TS dd449c402a034ed5bd3f060ff6c92899 (127.7.111.65:41927): tablet 18b436754fc645bb92728cc4ad61176f (table test-table [id=f548c9297c084063b025a978f6673d87]) successfully deleted
I20260811 12:51:27.034540 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 5503
I20260811 12:51:27.041286 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:27.035379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47606 (local address 127.7.111.66:34839)
0811 12:51:27.035576 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:27.035582 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:27.035601 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:51:27.035662 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:27.035666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:27.035778 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:27.035868 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:27.036304 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:27.037081 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:27.038401 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:27.038821 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:27.038932 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:27.039969 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:27.039977 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:27.039992 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:27.040570 (+ 578us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:27.040618 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:27.040643 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:27.040751 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:27.040877 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 12:51:27.040982 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:51:27.042137 5693 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:27.130402 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 5503
I20260811 12:51:27.153856 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 5705
I20260811 12:51:27.161090 5932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:27.154643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54688 (local address 127.7.111.67:46289)
0811 12:51:27.154840 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:27.154846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:27.154946 (+ 100us) server_negotiation.cc:407] Connection header received
0811 12:51:27.155124 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:27.155128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:27.155221 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:27.155307 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:27.155765 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:27.156553 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:27.157914 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:27.158370 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:27.158483 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:27.159529 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:27.159539 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:27.159555 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:27.160316 (+ 761us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:27.160370 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:27.160391 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:27.160497 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:27.160640 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 12:51:27.160757 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:51:27.161783 5904 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:27.281785 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 5705
I20260811 12:51:27.310504 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 5137
I20260811 12:51:27.311595 5234 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:27.419104 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 5137
2026-08-11T12:51:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3015 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:51:27 dist-test-slave-f0tn krb5kdc[5999](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:51:27 dist-test-slave-f0tn krb5kdc[5999](info): set up 2 sockets
Aug 11 12:51:27 dist-test-slave-f0tn krb5kdc[5999](info): commencing operation
krb5kdc: starting...
W20260811 12:51:28.737021 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:51:28 dist-test-slave-f0tn krb5kdc[5999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:51:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:51:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:51:29.046237 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:33327
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:33327
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:51:29.382737 6015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:29.383030 6015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:29.383160 6015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:29.390237 6015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:51:29.390338 6015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:29.390412 6015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:29.390486 6015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:51:29.390543 6015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:51:29.400344 6015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:33327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:33327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:29.401697 6015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:29.403048 6015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:29.414328 6023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:29.414361 6030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:29.414287 6026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:29.415282 6015 server_base.cc:1034] running on GCE node
I20260811 12:51:29.415970 6015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:29.417306 6015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:29.418530 6015 hybrid_clock.cc:648] HybridClock initialized: now 1786452689418483 us; error 34 us; skew 500 ppm
I20260811 12:51:29.418848 6015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:51:29 dist-test-slave-f0tn krb5kdc[5999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452689, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:51:29.426802 6015 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:51:29.428308 6015 webserver.cc:460] Webserver started at http://127.7.111.126:36479/ using document root <none> and password file <none>
I20260811 12:51:29.428977 6015 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:29.429102 6015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:29.429442 6015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:29.432627 6015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "a7133c51d9f24e88855ac375c8afe7ee"
format_stamp: "Formatted at 2026-08-11 12:51:29 on dist-test-slave-f0tn"
server_key: "143cbac8170740d1205d0793b507a94a"
server_key_iv: "e1f58749c8d8d9f499fb14e1f8f52011"
server_key_version: "encryptionkey@0"
I20260811 12:51:29.433435 6015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "a7133c51d9f24e88855ac375c8afe7ee"
format_stamp: "Formatted at 2026-08-11 12:51:29 on dist-test-slave-f0tn"
server_key: "143cbac8170740d1205d0793b507a94a"
server_key_iv: "e1f58749c8d8d9f499fb14e1f8f52011"
server_key_version: "encryptionkey@0"
I20260811 12:51:29.439034 6015 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 12:51:29.442636 6038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:29.443955 6015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:51:29.444142 6015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "a7133c51d9f24e88855ac375c8afe7ee"
format_stamp: "Formatted at 2026-08-11 12:51:29 on dist-test-slave-f0tn"
server_key: "143cbac8170740d1205d0793b507a94a"
server_key_iv: "e1f58749c8d8d9f499fb14e1f8f52011"
server_key_version: "encryptionkey@0"
I20260811 12:51:29.444342 6015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:29.462122 6015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:29.466079 6015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:29.466364 6015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:29.484726 6015 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:33327
I20260811 12:51:29.484733 6120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:33327 every 8 connection(s)
I20260811 12:51:29.486229 6015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:51:29.487581 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 6015
I20260811 12:51:29.487751 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:51:29.488296 7613 external_mini_cluster.cc:1428] Setting key 3e1690e23d2d6afb0a772db99f2d8360
I20260811 12:51:29.490829 6122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 12:51:29 dist-test-slave-f0tn krb5kdc[5999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:51:29.503506 6122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee: Bootstrap starting.
I20260811 12:51:29.507369 6122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:29.508688 6122 log.cc:824] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:29.511554 6122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee: No bootstrap required, opened a new log
I20260811 12:51:29.512620 6125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:29.490238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55504 (local address 127.7.111.126:33327)
0811 12:51:29.491074 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:29.491090 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:29.491139 (+ 49us) server_negotiation.cc:407] Connection header received
0811 12:51:29.492851 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:29.492873 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:29.493270 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:29.493680 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:29.495510 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:29.496964 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:29.498789 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:29.499529 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:29.499692 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:29.504549 (+ 4857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:29.504561 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:29.504579 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:29.504631 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:29.508385 (+ 3754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:29.509115 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:29.509122 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:29.509129 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:29.509332 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:29.509788 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:29.509792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:29.509794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:29.510332 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:29.510615 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:29.511096 (+ 481us) server_negotiation.cc:299] Negotiation successful
0811 12:51:29.511586 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":387,"threads_started":1}
I20260811 12:51:29.519038 6122 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7133c51d9f24e88855ac375c8afe7ee" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 33327 } }
I20260811 12:51:29.519374 6122 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:29.519471 6122 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7133c51d9f24e88855ac375c8afe7ee, State: Initialized, Role: FOLLOWER
I20260811 12:51:29.520053 6122 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7133c51d9f24e88855ac375c8afe7ee" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 33327 } }
I20260811 12:51:29.520278 6122 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:51:29.520411 6122 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:51:29.520576 6122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:29.522986 6122 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7133c51d9f24e88855ac375c8afe7ee" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 33327 } }
I20260811 12:51:29.523535 6122 leader_election.cc:302] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7133c51d9f24e88855ac375c8afe7ee; no voters:
I20260811 12:51:29.524277 6122 leader_election.cc:288] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:51:29.524425 6127 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:51:29.525171 6127 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee [term 1 LEADER]: Becoming Leader. State: Replica: a7133c51d9f24e88855ac375c8afe7ee, State: Running, Role: LEADER
I20260811 12:51:29.525838 6127 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7133c51d9f24e88855ac375c8afe7ee" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 33327 } }
I20260811 12:51:29.527482 6122 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:51:29.529079 6131 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7133c51d9f24e88855ac375c8afe7ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7133c51d9f24e88855ac375c8afe7ee" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 33327 } } }
I20260811 12:51:29.529036 6130 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7133c51d9f24e88855ac375c8afe7ee. Latest consensus state: current_term: 1 leader_uuid: "a7133c51d9f24e88855ac375c8afe7ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7133c51d9f24e88855ac375c8afe7ee" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 33327 } } }
I20260811 12:51:29.529474 6130 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:29.529476 6131 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:29.530500 6137 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:51:29.537395 6137 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:51:29.548940 6137 catalog_manager.cc:1347] Generated new cluster ID: 3c40a16298be43b7ac7b0d310618b020
I20260811 12:51:29.549088 6137 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:51:29.571671 6137 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:51:29.573647 6137 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:51:29.582501 6137 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee: Generated new TSK 0
I20260811 12:51:29.583496 6137 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.111.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
I20260811 12:51:29.654764 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:33327
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:51:29.991137 6164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:29.991387 6164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:29.991508 6164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:29.998768 6164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:29.998873 6164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:29.999024 6164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:51:30.008858 6164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:33327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:30.010380 6164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:30.011759 6164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:30.024943 6170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:30.024981 6171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:30.025023 6173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:30.025709 6164 server_base.cc:1034] running on GCE node
I20260811 12:51:30.026357 6164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:30.027326 6164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:30.028553 6164 hybrid_clock.cc:648] HybridClock initialized: now 1786452690028491 us; error 53 us; skew 500 ppm
I20260811 12:51:30.028928 6164 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:51:30 dist-test-slave-f0tn krb5kdc[5999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452690, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:51:30.032388 6164 init.cc:377] Logged in from keytab as kudu/127.7.111.65@KRBTEST.COM (short username kudu)
I20260811 12:51:30.033974 6164 webserver.cc:460] Webserver started at http://127.7.111.65:34555/ using document root <none> and password file <none>
I20260811 12:51:30.034730 6164 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:30.034868 6164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:30.035202 6164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:30.038364 6164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "a3571a39bcfd45e0bc369d7979be1371"
format_stamp: "Formatted at 2026-08-11 12:51:30 on dist-test-slave-f0tn"
server_key: "0a7eaa8eb3d038f475ca25359070e638"
server_key_iv: "1862b3da346d5d0c583a22c48894bbbe"
server_key_version: "encryptionkey@0"
I20260811 12:51:30.039144 6164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "a3571a39bcfd45e0bc369d7979be1371"
format_stamp: "Formatted at 2026-08-11 12:51:30 on dist-test-slave-f0tn"
server_key: "0a7eaa8eb3d038f475ca25359070e638"
server_key_iv: "1862b3da346d5d0c583a22c48894bbbe"
server_key_version: "encryptionkey@0"
I20260811 12:51:30.044251 6164 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 12:51:30.047935 6188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:30.049325 6164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:51:30.049544 6164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "a3571a39bcfd45e0bc369d7979be1371"
format_stamp: "Formatted at 2026-08-11 12:51:30 on dist-test-slave-f0tn"
server_key: "0a7eaa8eb3d038f475ca25359070e638"
server_key_iv: "1862b3da346d5d0c583a22c48894bbbe"
server_key_version: "encryptionkey@0"
I20260811 12:51:30.049731 6164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:30.066813 6164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:30.070920 6164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:30.071213 6164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:30.072186 6164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:51:30.073567 6164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:51:30.073642 6164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:30.073773 6164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:51:30.073825 6164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:30.103562 6164 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:43751
I20260811 12:51:30.103605 6387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:43751 every 8 connection(s)
I20260811 12:51:30.104846 6164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:51:30.106752 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 6164
I20260811 12:51:30.106918 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:51:30.107409 7613 external_mini_cluster.cc:1428] Setting key 205480a499fa12de5fe00f1fba5acc12
Aug 11 12:51:30 dist-test-slave-f0tn krb5kdc[5999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452690, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:51:30.126186 6392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:30.107399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:59065 (local address 127.7.111.126:33327)
0811 12:51:30.107889 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:30.107895 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:30.108953 (+ 1058us) server_negotiation.cc:407] Connection header received
0811 12:51:30.111379 (+ 2426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:30.111383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:30.111445 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:30.111535 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:30.113445 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:30.114118 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:30.116149 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:30.116549 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:30.116656 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:30.120591 (+ 3935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:30.120596 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:30.120599 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:30.120631 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:30.123089 (+ 2458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:30.123887 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:30.123890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:30.123892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:30.124001 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:30.124413 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:30.124415 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:30.124417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:30.124655 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:30.124764 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:30.125581 (+ 817us) server_negotiation.cc:299] Negotiation successful
0811 12:51:30.125812 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":220,"threads_started":1}
I20260811 12:51:30.127149 6393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:30.107966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:33327 (local address 127.7.111.65:59065)
0811 12:51:30.108796 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:30.108843 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:30.111085 (+ 2242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:30.111806 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:30.111816 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:30.112505 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:51:30.113274 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:30.113298 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:30.114279 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:30.114285 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:30.115977 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:30.115992 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:30.116780 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:30.116786 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:30.117057 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:30.117766 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:51:30.117801 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:51:30.120372 (+ 2571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:51:30.123223 (+ 2851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:51:30.123233 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:51:30.123249 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:51:30.123787 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:51:30.124089 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:51:30.124094 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:51:30.124097 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:51:30.124324 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:51:30.124765 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:51:30.124772 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:51:30.125160 (+ 388us) client_negotiation.cc:769] Sending connection context
0811 12:51:30.125425 (+ 265us) client_negotiation.cc:240] Negotiation successful
0811 12:51:30.125959 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":324,"threads_started":1}
I20260811 12:51:30.128813 6388 heartbeater.cc:347] Connected to a master server at 127.7.111.126:33327
I20260811 12:51:30.129105 6388 heartbeater.cc:464] Registering TS with master...
I20260811 12:51:30.129879 6388 heartbeater.cc:511] Master 127.7.111.126:33327 requested a full tablet report, sending...
I20260811 12:51:30.132275 6060 ts_manager.cc:194] Registered new tserver with Master: a3571a39bcfd45e0bc369d7979be1371 (127.7.111.65:43751)
I20260811 12:51:30.134317 6060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:59065
WARNING: no policy specified for kudu/127.7.111.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
I20260811 12:51:30.180406 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:33327
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:51:30.527422 6402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:30.527731 6402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:30.527899 6402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:30.535621 6402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:30.535737 6402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:30.535863 6402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:51:30.545711 6402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:33327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:30.547012 6402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:30.548509 6402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:30.562058 6408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:30.561952 6410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:30.562109 6412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:30.563199 6402 server_base.cc:1034] running on GCE node
I20260811 12:51:30.563819 6402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:30.564829 6402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:30.566075 6402 hybrid_clock.cc:648] HybridClock initialized: now 1786452690566017 us; error 46 us; skew 500 ppm
I20260811 12:51:30.566404 6402 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:51:30 dist-test-slave-f0tn krb5kdc[5999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452690, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:51:30.569684 6402 init.cc:377] Logged in from keytab as kudu/127.7.111.66@KRBTEST.COM (short username kudu)
I20260811 12:51:30.571254 6402 webserver.cc:460] Webserver started at http://127.7.111.66:36545/ using document root <none> and password file <none>
I20260811 12:51:30.571949 6402 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:30.572113 6402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:30.572456 6402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:30.575594 6402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "f872a2df2a5d4ea5a56ce27b8b0243ce"
format_stamp: "Formatted at 2026-08-11 12:51:30 on dist-test-slave-f0tn"
server_key: "c128f9e37a3c3b52318ae8251e305c7c"
server_key_iv: "bed27659ef2d513c5c336643035ea01b"
server_key_version: "encryptionkey@0"
I20260811 12:51:30.576434 6402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "f872a2df2a5d4ea5a56ce27b8b0243ce"
format_stamp: "Formatted at 2026-08-11 12:51:30 on dist-test-slave-f0tn"
server_key: "c128f9e37a3c3b52318ae8251e305c7c"
server_key_iv: "bed27659ef2d513c5c336643035ea01b"
server_key_version: "encryptionkey@0"
I20260811 12:51:30.581986 6402 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:51:30.585646 6427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:30.586947 6402 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:51:30.587111 6402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "f872a2df2a5d4ea5a56ce27b8b0243ce"
format_stamp: "Formatted at 2026-08-11 12:51:30 on dist-test-slave-f0tn"
server_key: "c128f9e37a3c3b52318ae8251e305c7c"
server_key_iv: "bed27659ef2d513c5c336643035ea01b"
server_key_version: "encryptionkey@0"
I20260811 12:51:30.587307 6402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:30.618405 6402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:30.622689 6402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:30.623024 6402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:30.624126 6402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:51:30.625592 6402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:51:30.625682 6402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:30.625828 6402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:51:30.625897 6402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:30.658574 6402 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:34837
I20260811 12:51:30.658609 6620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:34837 every 8 connection(s)
I20260811 12:51:30.659935 6402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:51:30.663472 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 6402
I20260811 12:51:30.663666 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:51:30.664237 7613 external_mini_cluster.cc:1428] Setting key eb02d3c9501611781ba0c20f341a7656
Aug 11 12:51:30 dist-test-slave-f0tn krb5kdc[5999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452690, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:51:30.679459 6624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:30.662544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:38889 (local address 127.7.111.126:33327)
0811 12:51:30.663016 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:30.663021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:30.663973 (+ 952us) server_negotiation.cc:407] Connection header received
0811 12:51:30.666206 (+ 2233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:30.666211 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:30.666291 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:30.666388 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:30.667865 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:30.668612 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:30.670225 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:30.670648 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:30.670767 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:30.674091 (+ 3324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:30.674096 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:30.674099 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:30.674131 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:30.676554 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:30.677234 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:30.677238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:30.677240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:30.677364 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:30.677803 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:30.677808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:30.677810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:30.678063 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:30.678193 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:30.678865 (+ 672us) server_negotiation.cc:299] Negotiation successful
0811 12:51:30.679085 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":257,"threads_started":1}
I20260811 12:51:30.680385 6625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:30.662974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:33327 (local address 127.7.111.66:38889)
0811 12:51:30.663827 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:30.663866 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:30.665912 (+ 2046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:30.666640 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:30.666648 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:30.667133 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:51:30.667700 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:30.667719 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:30.668764 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:30.668768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:30.670089 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:30.670102 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:30.670871 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:30.670878 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:30.671095 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:30.671723 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:51:30.671752 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:51:30.673879 (+ 2127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:51:30.676686 (+ 2807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:51:30.676694 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:51:30.676711 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:51:30.677111 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:51:30.677491 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:51:30.677496 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:51:30.677500 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:51:30.677688 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:51:30.678184 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:51:30.678191 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:51:30.678513 (+ 322us) client_negotiation.cc:769] Sending connection context
0811 12:51:30.678746 (+ 233us) client_negotiation.cc:240] Negotiation successful
0811 12:51:30.679259 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":336,"threads_started":1}
I20260811 12:51:30.681851 6621 heartbeater.cc:347] Connected to a master server at 127.7.111.126:33327
I20260811 12:51:30.682153 6621 heartbeater.cc:464] Registering TS with master...
I20260811 12:51:30.682879 6621 heartbeater.cc:511] Master 127.7.111.126:33327 requested a full tablet report, sending...
I20260811 12:51:30.684348 6060 ts_manager.cc:194] Registered new tserver with Master: f872a2df2a5d4ea5a56ce27b8b0243ce (127.7.111.66:34837)
I20260811 12:51:30.685441 6060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:38889
WARNING: no policy specified for kudu/127.7.111.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
I20260811 12:51:30.737278 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:33327
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:51:31.084676 6629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:31.085134 6629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:31.085266 6629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:31.093115 6629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:31.093254 6629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:31.093415 6629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:51:31.103940 6629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:33327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:31.105326 6629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:31.106887 6629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:31.120231 6635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:31.120179 6636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:31.120265 6644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:31.121235 6629 server_base.cc:1034] running on GCE node
I20260811 12:51:31.121898 6629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:31.122790 6629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:31.123986 6629 hybrid_clock.cc:648] HybridClock initialized: now 1786452691123937 us; error 44 us; skew 500 ppm
I20260811 12:51:31.124279 6629 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:51:31 dist-test-slave-f0tn krb5kdc[5999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452691, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:51:31.127408 6629 init.cc:377] Logged in from keytab as kudu/127.7.111.67@KRBTEST.COM (short username kudu)
I20260811 12:51:31.128861 6629 webserver.cc:460] Webserver started at http://127.7.111.67:38593/ using document root <none> and password file <none>
I20260811 12:51:31.129609 6629 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:31.129737 6629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:31.130043 6629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:31.132972 6629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "23e07d0ae757416d9ce402c7d6b51f93"
format_stamp: "Formatted at 2026-08-11 12:51:31 on dist-test-slave-f0tn"
server_key: "e0d2472857967d55007a018d14c4091c"
server_key_iv: "c0bc674222eee253e7671155c2025148"
server_key_version: "encryptionkey@0"
I20260811 12:51:31.133776 6629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "23e07d0ae757416d9ce402c7d6b51f93"
format_stamp: "Formatted at 2026-08-11 12:51:31 on dist-test-slave-f0tn"
server_key: "e0d2472857967d55007a018d14c4091c"
server_key_iv: "c0bc674222eee253e7671155c2025148"
server_key_version: "encryptionkey@0"
I20260811 12:51:31.137396 6388 heartbeater.cc:503] Master 127.7.111.126:33327 was elected leader, sending a full tablet report...
I20260811 12:51:31.139093 6629 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 12:51:31.142740 6656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:31.144093 6629 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:51:31.144268 6629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "23e07d0ae757416d9ce402c7d6b51f93"
format_stamp: "Formatted at 2026-08-11 12:51:31 on dist-test-slave-f0tn"
server_key: "e0d2472857967d55007a018d14c4091c"
server_key_iv: "c0bc674222eee253e7671155c2025148"
server_key_version: "encryptionkey@0"
I20260811 12:51:31.144479 6629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:31.170050 6629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:31.174871 6629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:31.175164 6629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:31.176213 6629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:51:31.177711 6629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:51:31.177798 6629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:31.177922 6629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:51:31.177984 6629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:31.209206 6629 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:40185
I20260811 12:51:31.209278 6860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:40185 every 8 connection(s)
I20260811 12:51:31.210541 6629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:51:31.219180 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 6629
I20260811 12:51:31.219384 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:51:31.219868 7613 external_mini_cluster.cc:1428] Setting key caf86d027dbc577f2a502ba73eee2336
Aug 11 12:51:31 dist-test-slave-f0tn krb5kdc[5999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452691, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:51:31.230788 6865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:31.213233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:53919 (local address 127.7.111.126:33327)
0811 12:51:31.213708 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:31.213714 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:31.214722 (+ 1008us) server_negotiation.cc:407] Connection header received
0811 12:51:31.216570 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:31.216575 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:31.216647 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:31.216748 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:31.218316 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.219315 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.221041 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.221656 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.221797 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:31.225422 (+ 3625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:31.225428 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:31.225432 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:31.225468 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:31.228101 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:31.228779 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:31.228783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:31.228785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:31.228911 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:31.229275 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:31.229278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:31.229280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:31.229575 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:31.229699 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:31.230351 (+ 652us) server_negotiation.cc:299] Negotiation successful
0811 12:51:31.230518 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":233,"threads_started":1}
I20260811 12:51:31.231813 6866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:31.213820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:33327 (local address 127.7.111.67:53919)
0811 12:51:31.214540 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:31.214585 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:31.216303 (+ 1718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:31.216944 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:31.216953 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:31.217556 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:51:31.218157 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:31.218175 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.219489 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.219495 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:31.220871 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:31.220887 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.221897 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.221904 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:31.222229 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:31.223021 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:51:31.223052 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:51:31.225183 (+ 2131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:51:31.228229 (+ 3046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:51:31.228237 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:51:31.228252 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:51:31.228697 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:51:31.228998 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:51:31.229001 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:51:31.229004 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:51:31.229187 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:51:31.229676 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:51:31.229681 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:51:31.230007 (+ 326us) client_negotiation.cc:769] Sending connection context
0811 12:51:31.230272 (+ 265us) client_negotiation.cc:240] Negotiation successful
0811 12:51:31.230685 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":265,"threads_started":1}
I20260811 12:51:31.233237 6861 heartbeater.cc:347] Connected to a master server at 127.7.111.126:33327
I20260811 12:51:31.233559 6861 heartbeater.cc:464] Registering TS with master...
I20260811 12:51:31.234344 6861 heartbeater.cc:511] Master 127.7.111.126:33327 requested a full tablet report, sending...
I20260811 12:51:31.235793 6060 ts_manager.cc:194] Registered new tserver with Master: 23e07d0ae757416d9ce402c7d6b51f93 (127.7.111.67:40185)
I20260811 12:51:31.236582 6060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:53919
I20260811 12:51:31.247434 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:51:31.263710 6865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:31.251684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55518 (local address 127.7.111.126:33327)
0811 12:51:31.251862 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:31.251867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:31.252344 (+ 477us) server_negotiation.cc:407] Connection header received
0811 12:51:31.252481 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:31.252484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:31.252562 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:31.252639 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:31.254151 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.254925 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.256106 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.256477 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.256561 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:31.259048 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:31.259053 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:31.259057 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:31.259083 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:31.261353 (+ 2270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:31.262068 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:31.262072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:31.262075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:31.262185 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:31.262547 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:31.262549 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:31.262551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:31.262847 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:31.263037 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:31.263342 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 12:51:31.263477 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:51:31.267462 6060 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:51:31.274344 6060 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:51:31.297402 6891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:31.286090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:34837 (local address 127.0.0.1:40244)
0811 12:51:31.286801 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:31.286847 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:31.287015 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:31.288307 (+ 1292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:31.288316 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:31.288360 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:31.288855 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:31.288870 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.291366 (+ 2496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.291373 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:31.294014 (+ 2641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:31.294031 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.296222 (+ 2191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.296229 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:31.296450 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:31.296510 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 12:51:31.296626 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 12:51:31.296840 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":1,"spinlock_wait_cycles":8064,"thread_start_us":291,"threads_started":1}
I20260811 12:51:31.298107 6893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:31.286376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:43751 (local address 127.0.0.1:44022)
0811 12:51:31.288252 (+ 1876us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:31.288273 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:31.288404 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:31.289315 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:31.289320 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:31.289347 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:31.289824 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:31.289835 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.292556 (+ 2721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.292563 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:31.294890 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:31.294907 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.297247 (+ 2340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.297257 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:31.297499 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:31.297536 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 12:51:31.297624 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 12:51:31.297774 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1731,"thread_start_us":1581,"threads_started":1}
I20260811 12:51:31.298972 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:31.286893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44022 (local address 127.7.111.65:43751)
0811 12:51:31.287639 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:31.287647 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:31.288547 (+ 900us) server_negotiation.cc:407] Connection header received
0811 12:51:31.288650 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:31.288657 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:31.288889 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:31.289077 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:31.290056 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.292377 (+ 2321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.295090 (+ 2713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.296669 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.296820 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:31.297081 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:31.298430 (+ 1349us) server_negotiation.cc:299] Negotiation successful
0811 12:51:31.298634 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":440,"threads_started":1}
I20260811 12:51:31.299911 6892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:31.286316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40244 (local address 127.7.111.66:34837)
0811 12:51:31.287631 (+ 1315us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:31.287640 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:31.287669 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:51:31.287771 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:31.287778 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:31.288031 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:31.289075 (+ 1044us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:31.289323 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.291200 (+ 1877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.294296 (+ 3096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.295941 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.299055 (+ 3114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:31.299302 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:31.299375 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 12:51:31.299554 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1169,"thread_start_us":1032,"threads_started":1}
I20260811 12:51:31.303351 6514 tablet_service.cc:1467] Processing CreateTablet for tablet d3f844f5aee74f1798569f1da07f4bdb (DEFAULT_TABLE table=test-table [id=9124d5c6c5604bd4af2a0fcc08ddfe28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:51:31.303351 6268 tablet_service.cc:1467] Processing CreateTablet for tablet d3f844f5aee74f1798569f1da07f4bdb (DEFAULT_TABLE table=test-table [id=9124d5c6c5604bd4af2a0fcc08ddfe28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:51:31.304848 6268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3f844f5aee74f1798569f1da07f4bdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:31.304867 6514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3f844f5aee74f1798569f1da07f4bdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:31.306573 6891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:31.288384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:40185 (local address 127.0.0.1:56218)
0811 12:51:31.297933 (+ 9549us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:31.297954 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:31.298063 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:31.301408 (+ 3345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:31.301412 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:31.301437 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:31.301591 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:31.301600 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.302983 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.302988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:31.304525 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:31.304545 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.305824 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.305830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:31.305997 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:31.306029 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:51:31.306118 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 12:51:31.306283 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9432,"thread_start_us":12552,"threads_started":1}
I20260811 12:51:31.306638 6900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:31.288606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56218 (local address 127.7.111.67:40185)
0811 12:51:31.300806 (+ 12200us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:31.300816 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:31.300850 (+ 34us) server_negotiation.cc:407] Connection header received
0811 12:51:31.300954 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:31.300962 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:31.301205 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:31.301397 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:31.301770 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.302792 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.304685 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.305673 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.305800 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:31.306029 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:31.306164 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 12:51:31.306331 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12011,"thread_start_us":11836,"threads_started":1}
I20260811 12:51:31.310118 6752 tablet_service.cc:1467] Processing CreateTablet for tablet d3f844f5aee74f1798569f1da07f4bdb (DEFAULT_TABLE table=test-table [id=9124d5c6c5604bd4af2a0fcc08ddfe28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:51:31.311379 6752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3f844f5aee74f1798569f1da07f4bdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:31.319854 6902 tablet_bootstrap.cc:492] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce: Bootstrap starting.
I20260811 12:51:31.320654 6901 tablet_bootstrap.cc:492] T d3f844f5aee74f1798569f1da07f4bdb P a3571a39bcfd45e0bc369d7979be1371: Bootstrap starting.
I20260811 12:51:31.324169 6903 tablet_bootstrap.cc:492] T d3f844f5aee74f1798569f1da07f4bdb P 23e07d0ae757416d9ce402c7d6b51f93: Bootstrap starting.
I20260811 12:51:31.324440 6902 tablet_bootstrap.cc:654] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:31.324669 6901 tablet_bootstrap.cc:654] T d3f844f5aee74f1798569f1da07f4bdb P a3571a39bcfd45e0bc369d7979be1371: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:31.325878 6902 log.cc:824] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:31.326434 6901 log.cc:824] T d3f844f5aee74f1798569f1da07f4bdb P a3571a39bcfd45e0bc369d7979be1371: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:31.327896 6903 tablet_bootstrap.cc:654] T d3f844f5aee74f1798569f1da07f4bdb P 23e07d0ae757416d9ce402c7d6b51f93: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:31.329221 6903 log.cc:824] T d3f844f5aee74f1798569f1da07f4bdb P 23e07d0ae757416d9ce402c7d6b51f93: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:31.329586 6902 tablet_bootstrap.cc:492] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce: No bootstrap required, opened a new log
I20260811 12:51:31.329790 6901 tablet_bootstrap.cc:492] T d3f844f5aee74f1798569f1da07f4bdb P a3571a39bcfd45e0bc369d7979be1371: No bootstrap required, opened a new log
I20260811 12:51:31.329866 6902 ts_tablet_manager.cc:1397] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 12:51:31.330086 6901 ts_tablet_manager.cc:1397] T d3f844f5aee74f1798569f1da07f4bdb P a3571a39bcfd45e0bc369d7979be1371: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 12:51:31.332576 6903 tablet_bootstrap.cc:492] T d3f844f5aee74f1798569f1da07f4bdb P 23e07d0ae757416d9ce402c7d6b51f93: No bootstrap required, opened a new log
I20260811 12:51:31.332844 6903 ts_tablet_manager.cc:1397] T d3f844f5aee74f1798569f1da07f4bdb P 23e07d0ae757416d9ce402c7d6b51f93: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260811 12:51:31.337872 6902 raft_consensus.cc:348] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f872a2df2a5d4ea5a56ce27b8b0243ce" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 34837 } } peers { permanent_uuid: "a3571a39bcfd45e0bc369d7979be1371" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43751 } } peers { permanent_uuid: "23e07d0ae757416d9ce402c7d6b51f93" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 40185 } }
I20260811 12:51:31.338250 6902 raft_consensus.cc:374] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:31.338342 6902 raft_consensus.cc:729] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f872a2df2a5d4ea5a56ce27b8b0243ce, State: Initialized, Role: FOLLOWER
I20260811 12:51:31.338928 6902 consensus_queue.cc:260] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f872a2df2a5d4ea5a56ce27b8b0243ce" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 34837 } } peers { permanent_uuid: "a3571a39bcfd45e0bc369d7979be1371" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43751 } } peers { permanent_uuid: "23e07d0ae757416d9ce402c7d6b51f93" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 40185 } }
I20260811 12:51:31.340010 6621 heartbeater.cc:503] Master 127.7.111.126:33327 was elected leader, sending a full tablet report...
I20260811 12:51:31.340292 6901 raft_consensus.cc:348] T d3f844f5aee74f1798569f1da07f4bdb P a3571a39bcfd45e0bc369d7979be1371 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f872a2df2a5d4ea5a56ce27b8b0243ce" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 34837 } } peers { permanent_uuid: "a3571a39bcfd45e0bc369d7979be1371" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43751 } } peers { permanent_uuid: "23e07d0ae757416d9ce402c7d6b51f93" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 40185 } }
I20260811 12:51:31.340710 6902 ts_tablet_manager.cc:1428] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260811 12:51:31.340713 6901 raft_consensus.cc:374] T d3f844f5aee74f1798569f1da07f4bdb P a3571a39bcfd45e0bc369d7979be1371 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:31.340802 6901 raft_consensus.cc:729] T d3f844f5aee74f1798569f1da07f4bdb P a3571a39bcfd45e0bc369d7979be1371 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3571a39bcfd45e0bc369d7979be1371, State: Initialized, Role: FOLLOWER
I20260811 12:51:31.341424 6901 consensus_queue.cc:260] T d3f844f5aee74f1798569f1da07f4bdb P a3571a39bcfd45e0bc369d7979be1371 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f872a2df2a5d4ea5a56ce27b8b0243ce" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 34837 } } peers { permanent_uuid: "a3571a39bcfd45e0bc369d7979be1371" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43751 } } peers { permanent_uuid: "23e07d0ae757416d9ce402c7d6b51f93" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 40185 } }
I20260811 12:51:31.343140 6903 raft_consensus.cc:348] T d3f844f5aee74f1798569f1da07f4bdb P 23e07d0ae757416d9ce402c7d6b51f93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f872a2df2a5d4ea5a56ce27b8b0243ce" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 34837 } } peers { permanent_uuid: "a3571a39bcfd45e0bc369d7979be1371" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43751 } } peers { permanent_uuid: "23e07d0ae757416d9ce402c7d6b51f93" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 40185 } }
I20260811 12:51:31.343587 6903 raft_consensus.cc:374] T d3f844f5aee74f1798569f1da07f4bdb P 23e07d0ae757416d9ce402c7d6b51f93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:31.343698 6903 raft_consensus.cc:729] T d3f844f5aee74f1798569f1da07f4bdb P 23e07d0ae757416d9ce402c7d6b51f93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23e07d0ae757416d9ce402c7d6b51f93, State: Initialized, Role: FOLLOWER
I20260811 12:51:31.344556 6901 ts_tablet_manager.cc:1428] T d3f844f5aee74f1798569f1da07f4bdb P a3571a39bcfd45e0bc369d7979be1371: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260811 12:51:31.344403 6903 consensus_queue.cc:260] T d3f844f5aee74f1798569f1da07f4bdb P 23e07d0ae757416d9ce402c7d6b51f93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f872a2df2a5d4ea5a56ce27b8b0243ce" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 34837 } } peers { permanent_uuid: "a3571a39bcfd45e0bc369d7979be1371" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43751 } } peers { permanent_uuid: "23e07d0ae757416d9ce402c7d6b51f93" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 40185 } }
I20260811 12:51:31.346150 6861 heartbeater.cc:503] Master 127.7.111.126:33327 was elected leader, sending a full tablet report...
I20260811 12:51:31.346822 6903 ts_tablet_manager.cc:1428] T d3f844f5aee74f1798569f1da07f4bdb P 23e07d0ae757416d9ce402c7d6b51f93: Time spent starting tablet: real 0.014s user 0.013s sys 0.002s
W20260811 12:51:31.357088 6391 tablet.cc:2369] T d3f844f5aee74f1798569f1da07f4bdb P a3571a39bcfd45e0bc369d7979be1371: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:51:31.411935 6623 tablet.cc:2369] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:51:31.462265 6862 tablet.cc:2369] T d3f844f5aee74f1798569f1da07f4bdb P 23e07d0ae757416d9ce402c7d6b51f93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:51:31.483453 6910 raft_consensus.cc:482] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:51:31.483757 6910 raft_consensus.cc:504] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f872a2df2a5d4ea5a56ce27b8b0243ce" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 34837 } } peers { permanent_uuid: "a3571a39bcfd45e0bc369d7979be1371" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43751 } } peers { permanent_uuid: "23e07d0ae757416d9ce402c7d6b51f93" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 40185 } }
I20260811 12:51:31.485615 6910 leader_election.cc:288] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3571a39bcfd45e0bc369d7979be1371 (127.7.111.65:43751), 23e07d0ae757416d9ce402c7d6b51f93 (127.7.111.67:40185)
I20260811 12:51:31.494120 6900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:31.486298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:42755 (local address 127.7.111.67:40185)
0811 12:51:31.486510 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:31.486517 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:31.488011 (+ 1494us) server_negotiation.cc:407] Connection header received
0811 12:51:31.488160 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:31.488165 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:31.488252 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:31.488334 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:31.488957 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.490065 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.492336 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.493099 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.493248 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:31.493326 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:31.493729 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 12:51:31.493867 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:51:31.494160 6915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:31.485871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:40185 (local address 127.7.111.66:42755)
0811 12:51:31.487902 (+ 2031us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:31.487920 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:31.488043 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:31.488377 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:31.488382 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:31.488407 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:31.488777 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:31.488789 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.490222 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.490227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:31.492191 (+ 1964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:31.492203 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.493314 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.493320 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:31.493542 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:31.493575 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:51:31.493671 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 12:51:31.493803 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1911,"thread_start_us":1376,"threads_started":1}
I20260811 12:51:31.494297 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:31.486846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:34217 (local address 127.7.111.65:43751)
0811 12:51:31.487053 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:31.487059 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:31.487727 (+ 668us) server_negotiation.cc:407] Connection header received
0811 12:51:31.488018 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:31.488023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:31.488101 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:31.488189 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:31.488842 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.489619 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.492032 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.493288 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.493451 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:31.493521 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:31.493877 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 12:51:31.494022 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 12:51:31.494802 6916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:31.485837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:43751 (local address 127.7.111.66:34217)
0811 12:51:31.487608 (+ 1771us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:31.487627 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:31.487852 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:31.488212 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:31.488217 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:31.488241 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:31.488677 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:31.488688 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.489747 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.489753 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:31.491846 (+ 2093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:31.491867 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.493519 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.493525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:31.493698 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:31.493727 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 12:51:31.494364 (+ 637us) client_negotiation.cc:240] Negotiation successful
0811 12:51:31.494491 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1623,"thread_start_us":1752,"threads_started":1}
I20260811 12:51:31.495570 6789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3f844f5aee74f1798569f1da07f4bdb" candidate_uuid: "f872a2df2a5d4ea5a56ce27b8b0243ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23e07d0ae757416d9ce402c7d6b51f93" is_pre_election: true
I20260811 12:51:31.495916 6311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3f844f5aee74f1798569f1da07f4bdb" candidate_uuid: "f872a2df2a5d4ea5a56ce27b8b0243ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3571a39bcfd45e0bc369d7979be1371" is_pre_election: true
I20260811 12:51:31.496186 6789 raft_consensus.cc:2454] T d3f844f5aee74f1798569f1da07f4bdb P 23e07d0ae757416d9ce402c7d6b51f93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f872a2df2a5d4ea5a56ce27b8b0243ce in term 0.
I20260811 12:51:31.496484 6311 raft_consensus.cc:2454] T d3f844f5aee74f1798569f1da07f4bdb P a3571a39bcfd45e0bc369d7979be1371 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f872a2df2a5d4ea5a56ce27b8b0243ce in term 0.
I20260811 12:51:31.496954 6430 leader_election.cc:302] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23e07d0ae757416d9ce402c7d6b51f93, f872a2df2a5d4ea5a56ce27b8b0243ce; no voters:
I20260811 12:51:31.497400 6910 raft_consensus.cc:2781] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:51:31.497558 6910 raft_consensus.cc:482] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:51:31.497658 6910 raft_consensus.cc:3037] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:31.499805 6910 raft_consensus.cc:504] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f872a2df2a5d4ea5a56ce27b8b0243ce" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 34837 } } peers { permanent_uuid: "a3571a39bcfd45e0bc369d7979be1371" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43751 } } peers { permanent_uuid: "23e07d0ae757416d9ce402c7d6b51f93" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 40185 } }
I20260811 12:51:31.500428 6910 leader_election.cc:288] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce [CANDIDATE]: Term 1 election: Requested vote from peers a3571a39bcfd45e0bc369d7979be1371 (127.7.111.65:43751), 23e07d0ae757416d9ce402c7d6b51f93 (127.7.111.67:40185)
I20260811 12:51:31.500806 6311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3f844f5aee74f1798569f1da07f4bdb" candidate_uuid: "f872a2df2a5d4ea5a56ce27b8b0243ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3571a39bcfd45e0bc369d7979be1371"
I20260811 12:51:31.500970 6789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3f844f5aee74f1798569f1da07f4bdb" candidate_uuid: "f872a2df2a5d4ea5a56ce27b8b0243ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23e07d0ae757416d9ce402c7d6b51f93"
I20260811 12:51:31.501025 6311 raft_consensus.cc:3037] T d3f844f5aee74f1798569f1da07f4bdb P a3571a39bcfd45e0bc369d7979be1371 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:31.501186 6789 raft_consensus.cc:3037] T d3f844f5aee74f1798569f1da07f4bdb P 23e07d0ae757416d9ce402c7d6b51f93 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:31.504287 6311 raft_consensus.cc:2454] T d3f844f5aee74f1798569f1da07f4bdb P a3571a39bcfd45e0bc369d7979be1371 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f872a2df2a5d4ea5a56ce27b8b0243ce in term 1.
I20260811 12:51:31.504287 6789 raft_consensus.cc:2454] T d3f844f5aee74f1798569f1da07f4bdb P 23e07d0ae757416d9ce402c7d6b51f93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f872a2df2a5d4ea5a56ce27b8b0243ce in term 1.
I20260811 12:51:31.504691 6430 leader_election.cc:302] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23e07d0ae757416d9ce402c7d6b51f93, f872a2df2a5d4ea5a56ce27b8b0243ce; no voters:
I20260811 12:51:31.504959 6910 raft_consensus.cc:2781] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:51:31.505455 6910 raft_consensus.cc:686] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce [term 1 LEADER]: Becoming Leader. State: Replica: f872a2df2a5d4ea5a56ce27b8b0243ce, State: Running, Role: LEADER
I20260811 12:51:31.505923 6910 consensus_queue.cc:237] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f872a2df2a5d4ea5a56ce27b8b0243ce" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 34837 } } peers { permanent_uuid: "a3571a39bcfd45e0bc369d7979be1371" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43751 } } peers { permanent_uuid: "23e07d0ae757416d9ce402c7d6b51f93" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 40185 } }
I20260811 12:51:31.510542 6060 catalog_manager.cc:5495] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce reported cstate change: term changed from 0 to 1, leader changed from <none> to f872a2df2a5d4ea5a56ce27b8b0243ce (127.7.111.66). New cstate: current_term: 1 leader_uuid: "f872a2df2a5d4ea5a56ce27b8b0243ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f872a2df2a5d4ea5a56ce27b8b0243ce" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 34837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3571a39bcfd45e0bc369d7979be1371" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23e07d0ae757416d9ce402c7d6b51f93" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 40185 } health_report { overall_health: UNKNOWN } } }
I20260811 12:51:31.536612 6892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:31.530886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40252 (local address 127.7.111.66:34837)
0811 12:51:31.531129 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:31.531137 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:31.531354 (+ 217us) server_negotiation.cc:407] Connection header received
0811 12:51:31.531581 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:31.531586 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:31.531665 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:31.531796 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:51:31.532216 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.532951 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.534410 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.534829 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.534940 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:31.535515 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:51:31.536005 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:51:31.536161 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:31.536328 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 12:51:31.536376 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:51:31.544699 6311 raft_consensus.cc:1261] T d3f844f5aee74f1798569f1da07f4bdb P a3571a39bcfd45e0bc369d7979be1371 [term 1 FOLLOWER]: Refusing update from remote peer f872a2df2a5d4ea5a56ce27b8b0243ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:51:31.544697 6789 raft_consensus.cc:1261] T d3f844f5aee74f1798569f1da07f4bdb P 23e07d0ae757416d9ce402c7d6b51f93 [term 1 FOLLOWER]: Refusing update from remote peer f872a2df2a5d4ea5a56ce27b8b0243ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:51:31.545794 6917 consensus_queue.cc:1035] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "23e07d0ae757416d9ce402c7d6b51f93" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 40185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:51:31.546129 6910 consensus_queue.cc:1035] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3571a39bcfd45e0bc369d7979be1371" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 43751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:51:31.570307 6060 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:51:31.570619 6060 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:51:31.574527 6060 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a7133c51d9f24e88855ac375c8afe7ee: Sending DeleteTablet for 3 replicas of tablet d3f844f5aee74f1798569f1da07f4bdb
I20260811 12:51:31.575510 6514 tablet_service.cc:1514] Processing DeleteTablet for tablet d3f844f5aee74f1798569f1da07f4bdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:51:31 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:40244
I20260811 12:51:31.575803 6268 tablet_service.cc:1514] Processing DeleteTablet for tablet d3f844f5aee74f1798569f1da07f4bdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:51:31 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:44022
I20260811 12:51:31.575814 6752 tablet_service.cc:1514] Processing DeleteTablet for tablet d3f844f5aee74f1798569f1da07f4bdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:51:31 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:56218
I20260811 12:51:31.576040 6750 tablet_service.cc:1514] Processing DeleteTablet for tablet d3f844f5aee74f1798569f1da07f4bdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:51:31 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:56218
I20260811 12:51:31.576843 6936 tablet_replica.cc:330] T d3f844f5aee74f1798569f1da07f4bdb P a3571a39bcfd45e0bc369d7979be1371: stopping tablet replica
I20260811 12:51:31.577263 6936 raft_consensus.cc:2229] T d3f844f5aee74f1798569f1da07f4bdb P a3571a39bcfd45e0bc369d7979be1371 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:51:31.577797 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 6164
I20260811 12:51:31.577803 6936 raft_consensus.cc:2258] T d3f844f5aee74f1798569f1da07f4bdb P a3571a39bcfd45e0bc369d7979be1371 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:51:31.584173 6936 ts_tablet_manager.cc:1905] T d3f844f5aee74f1798569f1da07f4bdb P a3571a39bcfd45e0bc369d7979be1371: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:51:31.589558 6934 tablet_replica.cc:330] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce: stopping tablet replica
I20260811 12:51:31.590000 6934 raft_consensus.cc:2229] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:51:31.590771 6934 raft_consensus.cc:2258] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:51:31.592001 6936 ts_tablet_manager.cc:1918] T d3f844f5aee74f1798569f1da07f4bdb P a3571a39bcfd45e0bc369d7979be1371: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:51:31.592190 6936 log.cc:1196] T d3f844f5aee74f1798569f1da07f4bdb P a3571a39bcfd45e0bc369d7979be1371: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/d3f844f5aee74f1798569f1da07f4bdb
I20260811 12:51:31.592753 6936 ts_tablet_manager.cc:1939] T d3f844f5aee74f1798569f1da07f4bdb P a3571a39bcfd45e0bc369d7979be1371: Deleting consensus metadata
I20260811 12:51:31.593056 6934 ts_tablet_manager.cc:1905] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:51:31.594635 6042 catalog_manager.cc:4841] TS a3571a39bcfd45e0bc369d7979be1371 (127.7.111.65:43751): tablet d3f844f5aee74f1798569f1da07f4bdb (table test-table [id=9124d5c6c5604bd4af2a0fcc08ddfe28]) successfully deleted
I20260811 12:51:31.600632 6934 ts_tablet_manager.cc:1918] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:51:31.600819 6934 log.cc:1196] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/d3f844f5aee74f1798569f1da07f4bdb
I20260811 12:51:31.601882 6935 tablet_replica.cc:330] T d3f844f5aee74f1798569f1da07f4bdb P 23e07d0ae757416d9ce402c7d6b51f93: stopping tablet replica
I20260811 12:51:31.602258 6935 raft_consensus.cc:2229] T d3f844f5aee74f1798569f1da07f4bdb P 23e07d0ae757416d9ce402c7d6b51f93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:51:31.602487 6934 ts_tablet_manager.cc:1939] T d3f844f5aee74f1798569f1da07f4bdb P f872a2df2a5d4ea5a56ce27b8b0243ce: Deleting consensus metadata
I20260811 12:51:31.602770 6935 raft_consensus.cc:2258] T d3f844f5aee74f1798569f1da07f4bdb P 23e07d0ae757416d9ce402c7d6b51f93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:51:31.604445 6935 ts_tablet_manager.cc:1905] T d3f844f5aee74f1798569f1da07f4bdb P 23e07d0ae757416d9ce402c7d6b51f93: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 12:51:31 dist-test-slave-f0tn krb5kdc[5999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.65@KRBTEST.COM
I20260811 12:51:31.607075 6040 catalog_manager.cc:4841] TS f872a2df2a5d4ea5a56ce27b8b0243ce (127.7.111.66:34837): tablet d3f844f5aee74f1798569f1da07f4bdb (table test-table [id=9124d5c6c5604bd4af2a0fcc08ddfe28]) successfully deleted
I20260811 12:51:31.611040 6935 ts_tablet_manager.cc:1918] T d3f844f5aee74f1798569f1da07f4bdb P 23e07d0ae757416d9ce402c7d6b51f93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:51:31.611208 6935 log.cc:1196] T d3f844f5aee74f1798569f1da07f4bdb P 23e07d0ae757416d9ce402c7d6b51f93: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/d3f844f5aee74f1798569f1da07f4bdb
I20260811 12:51:31.611622 6935 ts_tablet_manager.cc:1939] T d3f844f5aee74f1798569f1da07f4bdb P 23e07d0ae757416d9ce402c7d6b51f93: Deleting consensus metadata
I20260811 12:51:31.613354 6039 catalog_manager.cc:4841] TS 23e07d0ae757416d9ce402c7d6b51f93 (127.7.111.67:40185): tablet d3f844f5aee74f1798569f1da07f4bdb (table test-table [id=9124d5c6c5604bd4af2a0fcc08ddfe28]) successfully deleted
W20260811 12:51:31.613852 6039 catalog_manager.cc:4808] TS 23e07d0ae757416d9ce402c7d6b51f93 (127.7.111.67:40185): delete failed for tablet d3f844f5aee74f1798569f1da07f4bdb because the tablet was not found. No further retry: Not found: Tablet not found: d3f844f5aee74f1798569f1da07f4bdb
I20260811 12:51:31.614814 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:31.579238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44024 (local address 127.7.111.65:43751)
0811 12:51:31.579398 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:31.579404 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:31.592599 (+ 13195us) server_negotiation.cc:407] Connection header received
0811 12:51:31.592978 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:31.592984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:31.593079 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:31.593163 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:31.597908 (+ 4745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.598988 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.600915 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.601557 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.601654 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:31.607189 (+ 5535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:31.607218 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:31.607237 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:31.607284 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:31.611004 (+ 3720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:31.611918 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:31.611927 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:31.611930 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:31.612094 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:31.612525 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:31.612530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:31.612533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:31.612988 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:31.613763 (+ 775us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:31.614249 (+ 486us) server_negotiation.cc:299] Negotiation successful
0811 12:51:31.614458 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 12:51:31.615447 6382 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:31.720281 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 6164
I20260811 12:51:31.743777 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 6402
Aug 11 12:51:31 dist-test-slave-f0tn krb5kdc[5999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.66@KRBTEST.COM
I20260811 12:51:31.759629 6892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:31.744562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40258 (local address 127.7.111.66:34837)
0811 12:51:31.744771 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:31.744777 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:31.744796 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:51:31.744950 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:31.744956 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:31.745029 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:31.745148 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:31.746479 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.747364 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.748717 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.749211 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.749320 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:31.753825 (+ 4505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:31.753837 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:31.753853 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:31.753886 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:31.756853 (+ 2967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:31.757536 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:31.757546 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:31.757549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:31.757710 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:31.758098 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:31.758102 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:31.758104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:31.758438 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:31.758759 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:31.759067 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 12:51:31.759308 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 12:51:31.760324 6613 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:31.876863 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 6402
I20260811 12:51:31.902788 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 6629
Aug 11 12:51:31 dist-test-slave-f0tn krb5kdc[5999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.67@KRBTEST.COM
I20260811 12:51:31.917155 6900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:31.903535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56232 (local address 127.7.111.67:40185)
0811 12:51:31.903699 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:31.903704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:31.903723 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:51:31.903879 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:31.903883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:31.903960 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:31.904043 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:31.905293 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.906039 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.907324 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:31.907688 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:31.907773 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:31.911933 (+ 4160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:31.911948 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:31.911966 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:31.912003 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:31.914598 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:31.915253 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:31.915264 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:31.915268 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:31.915466 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:31.915886 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:31.915890 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:31.915893 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:31.916272 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:31.916486 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:31.916730 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 12:51:31.916896 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:51:31.917820 6853 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:32.006709 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 6629
I20260811 12:51:32.031884 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 6015
I20260811 12:51:32.033017 6113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:32.142590 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 6015
2026-08-11T12:51:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4725 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T12:51:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:51:32Z Disabled control of system clock
I20260811 12:51:32.249132 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:40949
--webserver_interface=127.7.111.126
--webserver_port=0
--builtin_ntp_servers=127.7.111.84:33125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:40949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:51:32.583294 6949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:32.583567 6949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:32.583691 6949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:32.590755 6949 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:51:32.590859 6949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:32.590921 6949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:32.590997 6949 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:51:32.591056 6949 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:51:32.600597 6949 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:33125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:40949
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:40949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:32.602041 6949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:32.603935 6949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:32.615799 6962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:32.615850 6966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:32.615888 6961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:32.616437 6949 server_base.cc:1034] running on GCE node
I20260811 12:51:32.617149 6949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:32.618539 6949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:32.619796 6949 hybrid_clock.cc:648] HybridClock initialized: now 1786452692619742 us; error 43 us; skew 500 ppm
I20260811 12:51:32.620134 6949 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:51:32.622380 6949 webserver.cc:460] Webserver started at http://127.7.111.126:46477/ using document root <none> and password file <none>
I20260811 12:51:32.623039 6949 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:32.623131 6949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:32.623453 6949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:32.626331 6949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "18b1a008752b434aa4ede1c8f816a996"
format_stamp: "Formatted at 2026-08-11 12:51:32 on dist-test-slave-f0tn"
server_key: "a804a66afb55c204dc40f1fbde878872"
server_key_iv: "0cd6af62fec6870f5d13434d2e098e7d"
server_key_version: "encryptionkey@0"
I20260811 12:51:32.627107 6949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "18b1a008752b434aa4ede1c8f816a996"
format_stamp: "Formatted at 2026-08-11 12:51:32 on dist-test-slave-f0tn"
server_key: "a804a66afb55c204dc40f1fbde878872"
server_key_iv: "0cd6af62fec6870f5d13434d2e098e7d"
server_key_version: "encryptionkey@0"
I20260811 12:51:32.631944 6949 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 12:51:32.635169 6974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:32.636380 6949 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260811 12:51:32.636539 6949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "18b1a008752b434aa4ede1c8f816a996"
format_stamp: "Formatted at 2026-08-11 12:51:32 on dist-test-slave-f0tn"
server_key: "a804a66afb55c204dc40f1fbde878872"
server_key_iv: "0cd6af62fec6870f5d13434d2e098e7d"
server_key_version: "encryptionkey@0"
I20260811 12:51:32.636729 6949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:32.651866 6949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:32.652680 6949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:32.652951 6949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:32.670855 6949 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:40949
I20260811 12:51:32.670873 7066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:40949 every 8 connection(s)
I20260811 12:51:32.672341 6949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:51:32.676169 7067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:32.678113 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 6949
I20260811 12:51:32.678326 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:51:32.678872 7613 external_mini_cluster.cc:1428] Setting key 822e8c40d17fe82ef66adbd1f4ada258
I20260811 12:51:32.688259 7067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996: Bootstrap starting.
I20260811 12:51:32.691656 7067 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:32.692399 7070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:32.680813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58520 (local address 127.7.111.126:40949)
0811 12:51:32.681927 (+ 1114us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:32.681940 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:32.681986 (+ 46us) server_negotiation.cc:407] Connection header received
0811 12:51:32.683655 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:32.683685 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:32.684119 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:32.684423 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:32.685000 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:32.686382 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:32.688179 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:32.688862 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:32.689000 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:32.690173 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:32.690181 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:32.690193 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:32.690709 (+ 516us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:32.690802 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:32.690823 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:32.690910 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:32.691128 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 12:51:32.691429 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":413,"threads_started":1}
I20260811 12:51:32.692736 7067 log.cc:824] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:32.695595 7067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996: No bootstrap required, opened a new log
I20260811 12:51:32.702802 7067 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b1a008752b434aa4ede1c8f816a996" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40949 } }
I20260811 12:51:32.703133 7067 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:32.703231 7067 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18b1a008752b434aa4ede1c8f816a996, State: Initialized, Role: FOLLOWER
I20260811 12:51:32.703765 7067 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b1a008752b434aa4ede1c8f816a996" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40949 } }
I20260811 12:51:32.703984 7067 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:51:32.704118 7067 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:51:32.704267 7067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:32.706418 7067 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b1a008752b434aa4ede1c8f816a996" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40949 } }
I20260811 12:51:32.706951 7067 leader_election.cc:302] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18b1a008752b434aa4ede1c8f816a996; no voters:
I20260811 12:51:32.707616 7067 leader_election.cc:288] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:51:32.707760 7072 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:51:32.708446 7072 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996 [term 1 LEADER]: Becoming Leader. State: Replica: 18b1a008752b434aa4ede1c8f816a996, State: Running, Role: LEADER
I20260811 12:51:32.708937 7072 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b1a008752b434aa4ede1c8f816a996" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40949 } }
I20260811 12:51:32.709561 7067 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:51:32.711850 7074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18b1a008752b434aa4ede1c8f816a996. Latest consensus state: current_term: 1 leader_uuid: "18b1a008752b434aa4ede1c8f816a996" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b1a008752b434aa4ede1c8f816a996" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40949 } } }
I20260811 12:51:32.711706 7073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18b1a008752b434aa4ede1c8f816a996" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b1a008752b434aa4ede1c8f816a996" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40949 } } }
I20260811 12:51:32.712714 7074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996 [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:32.712796 7073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996 [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:32.713707 7086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:51:32.719101 7086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:51:32.731406 7086 catalog_manager.cc:1347] Generated new cluster ID: 8dc8190bd62b40f29bad417c55c826a6
I20260811 12:51:32.731513 7086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:51:32.747982 7086 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:51:32.749797 7086 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:51:32.769098 7086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996: Generated new TSK 0
I20260811 12:51:32.770205 7086 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:51:32.779742 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40949
--builtin_ntp_servers=127.7.111.84:33125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:51:33.123081 7105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:33.123375 7105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:33.123463 7105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:33.130824 7105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:33.130927 7105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:33.131062 7105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:51:33.140374 7105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:33125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40949
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:33.141845 7105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:33.143710 7105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:33.156409 7112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:33.156703 7115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:33.156773 7113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:33.157223 7105 server_base.cc:1034] running on GCE node
I20260811 12:51:33.157876 7105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:33.158798 7105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:33.160037 7105 hybrid_clock.cc:648] HybridClock initialized: now 1786452693159991 us; error 31 us; skew 500 ppm
I20260811 12:51:33.160367 7105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:51:33.162572 7105 webserver.cc:460] Webserver started at http://127.7.111.65:37351/ using document root <none> and password file <none>
I20260811 12:51:33.163244 7105 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:33.163343 7105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:33.163643 7105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:33.166471 7105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "f2d7d98800684ca4bf9caa6c899e8954"
format_stamp: "Formatted at 2026-08-11 12:51:33 on dist-test-slave-f0tn"
server_key: "62807eebd77763a1d91fa07ab793ae32"
server_key_iv: "9ac8c804d60aab53d65165d340104d59"
server_key_version: "encryptionkey@0"
I20260811 12:51:33.167229 7105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "f2d7d98800684ca4bf9caa6c899e8954"
format_stamp: "Formatted at 2026-08-11 12:51:33 on dist-test-slave-f0tn"
server_key: "62807eebd77763a1d91fa07ab793ae32"
server_key_iv: "9ac8c804d60aab53d65165d340104d59"
server_key_version: "encryptionkey@0"
I20260811 12:51:33.172134 7105 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 12:51:33.175602 7123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:33.176914 7105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:51:33.177067 7105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "f2d7d98800684ca4bf9caa6c899e8954"
format_stamp: "Formatted at 2026-08-11 12:51:33 on dist-test-slave-f0tn"
server_key: "62807eebd77763a1d91fa07ab793ae32"
server_key_iv: "9ac8c804d60aab53d65165d340104d59"
server_key_version: "encryptionkey@0"
I20260811 12:51:33.177275 7105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:33.190785 7105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:33.191625 7105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:33.191916 7105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:33.192894 7105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:51:33.194473 7105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:51:33.194572 7105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:33.194684 7105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:51:33.194763 7105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:33.227929 7105 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:45045
I20260811 12:51:33.227937 7313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:45045 every 8 connection(s)
I20260811 12:51:33.229774 7105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:51:33.233692 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 7105
I20260811 12:51:33.233883 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:51:33.234411 7613 external_mini_cluster.cc:1428] Setting key 48aa54c1fd5d498bf3358a509db98418
I20260811 12:51:33.237673 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40949
--builtin_ntp_servers=127.7.111.84:33125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:51:33.245223 7318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:33.232588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:57343 (local address 127.7.111.126:40949)
0811 12:51:33.233040 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:33.233045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:33.234607 (+ 1562us) server_negotiation.cc:407] Connection header received
0811 12:51:33.236779 (+ 2172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:33.236786 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:33.236871 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:33.236981 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:33.238413 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:33.239546 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:33.241855 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:33.242518 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:33.242644 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:33.244228 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:33.244235 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:33.244240 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:33.244265 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:33.244317 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:33.244334 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:33.244423 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:33.244650 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 12:51:33.244765 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":225,"threads_started":1}
I20260811 12:51:33.246344 7319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:33.233271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40949 (local address 127.7.111.65:57343)
0811 12:51:33.234428 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:33.234478 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:33.236476 (+ 1998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:33.237256 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:33.237267 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:33.237410 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:51:33.238228 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:33.238253 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:33.239810 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:33.239816 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:33.241678 (+ 1862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:33.241693 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:33.242750 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:33.242756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:33.243016 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:33.243860 (+ 844us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:51:33.243864 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:51:33.243916 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:51:33.244015 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:51:33.244432 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:51:33.244443 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:51:33.244450 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 12:51:33.244619 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 12:51:33.245002 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":332,"threads_started":1}
I20260811 12:51:33.248020 7316 heartbeater.cc:347] Connected to a master server at 127.7.111.126:40949
I20260811 12:51:33.248360 7316 heartbeater.cc:464] Registering TS with master...
I20260811 12:51:33.249311 7316 heartbeater.cc:511] Master 127.7.111.126:40949 requested a full tablet report, sending...
I20260811 12:51:33.251748 7009 ts_manager.cc:194] Registered new tserver with Master: f2d7d98800684ca4bf9caa6c899e8954 (127.7.111.65:45045)
I20260811 12:51:33.253648 7009 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.111.65:57343
W20260811 12:51:33.579041 7320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:33.579306 7320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:33.579434 7320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:33.586555 7320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:33.586647 7320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:33.586798 7320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:51:33.595721 7320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:33125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40949
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:33.597011 7320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:33.598898 7320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:33.611809 7328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:33.611840 7329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:33.611867 7332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:33.612442 7320 server_base.cc:1034] running on GCE node
I20260811 12:51:33.613026 7320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:33.613997 7320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:33.615209 7320 hybrid_clock.cc:648] HybridClock initialized: now 1786452693615141 us; error 57 us; skew 500 ppm
I20260811 12:51:33.615525 7320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:51:33.617734 7320 webserver.cc:460] Webserver started at http://127.7.111.66:35631/ using document root <none> and password file <none>
I20260811 12:51:33.618402 7320 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:33.618495 7320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:33.618865 7320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:33.621727 7320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "d4c1ffdce6e74d72a35ff2d729fcf57a"
format_stamp: "Formatted at 2026-08-11 12:51:33 on dist-test-slave-f0tn"
server_key: "1aeb23834f4b173d40780866a0ae1d8a"
server_key_iv: "89a22d351c7752db6b8e06796237509e"
server_key_version: "encryptionkey@0"
I20260811 12:51:33.622505 7320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "d4c1ffdce6e74d72a35ff2d729fcf57a"
format_stamp: "Formatted at 2026-08-11 12:51:33 on dist-test-slave-f0tn"
server_key: "1aeb23834f4b173d40780866a0ae1d8a"
server_key_iv: "89a22d351c7752db6b8e06796237509e"
server_key_version: "encryptionkey@0"
I20260811 12:51:33.627516 7320 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 12:51:33.630997 7341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:33.632244 7320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:51:33.632395 7320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "d4c1ffdce6e74d72a35ff2d729fcf57a"
format_stamp: "Formatted at 2026-08-11 12:51:33 on dist-test-slave-f0tn"
server_key: "1aeb23834f4b173d40780866a0ae1d8a"
server_key_iv: "89a22d351c7752db6b8e06796237509e"
server_key_version: "encryptionkey@0"
I20260811 12:51:33.632586 7320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:33.655773 7320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:33.656522 7320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:33.656813 7320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:33.657827 7320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:51:33.668442 7320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:51:33.668555 7320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:33.668664 7320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:51:33.668741 7320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:33.704454 7320 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:37337
I20260811 12:51:33.704572 7523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:37337 every 8 connection(s)
I20260811 12:51:33.705790 7320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:51:33.715178 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 7320
I20260811 12:51:33.715363 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:51:33.715871 7613 external_mini_cluster.cc:1428] Setting key 30c109a965613d176a52224c8a8437a0
I20260811 12:51:33.719116 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40949
--builtin_ntp_servers=127.7.111.84:33125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:51:33.726203 7318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:33.709577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:35099 (local address 127.7.111.126:40949)
0811 12:51:33.713568 (+ 3991us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:33.713575 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:33.714438 (+ 863us) server_negotiation.cc:407] Connection header received
0811 12:51:33.717921 (+ 3483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:33.717928 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:33.718015 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:33.718087 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:33.719857 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:33.720831 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:33.723219 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:33.723733 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:33.723849 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:33.725362 (+ 1513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:33.725368 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:33.725372 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:33.725423 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:33.725460 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:33.725475 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:33.725557 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:33.725822 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 12:51:33.725921 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3831}
I20260811 12:51:33.727465 7529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:33.709271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40949 (local address 127.7.111.66:35099)
0811 12:51:33.714261 (+ 4990us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:33.714316 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:33.717557 (+ 3241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:33.718644 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:33.718656 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:33.718809 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:51:33.719641 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:33.719670 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:33.721024 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:33.721030 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:33.723060 (+ 2030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:33.723076 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:33.723947 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:33.723954 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:33.724237 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:33.725004 (+ 767us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:51:33.725008 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:51:33.725061 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:51:33.725160 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:51:33.725592 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:51:33.725604 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:51:33.725612 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:51:33.725802 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 12:51:33.726192 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4410,"thread_start_us":3875,"threads_started":1}
I20260811 12:51:33.728914 7525 heartbeater.cc:347] Connected to a master server at 127.7.111.126:40949
I20260811 12:51:33.729239 7525 heartbeater.cc:464] Registering TS with master...
I20260811 12:51:33.730134 7525 heartbeater.cc:511] Master 127.7.111.126:40949 requested a full tablet report, sending...
I20260811 12:51:33.731662 7009 ts_manager.cc:194] Registered new tserver with Master: d4c1ffdce6e74d72a35ff2d729fcf57a (127.7.111.66:37337)
I20260811 12:51:33.732772 7009 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.111.66:35099
W20260811 12:51:34.094871 7530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:34.095158 7530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:34.095304 7530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:34.102735 7530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:34.102866 7530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:34.103021 7530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:51:34.112285 7530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:33125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:40949
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:34.113673 7530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:34.115444 7530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:34.128118 7539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:34.128054 7547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:34.128082 7543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:34.129091 7530 server_base.cc:1034] running on GCE node
I20260811 12:51:34.129719 7530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:34.130733 7530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:34.131935 7530 hybrid_clock.cc:648] HybridClock initialized: now 1786452694131888 us; error 39 us; skew 500 ppm
I20260811 12:51:34.132262 7530 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:51:34.134404 7530 webserver.cc:460] Webserver started at http://127.7.111.67:44095/ using document root <none> and password file <none>
I20260811 12:51:34.135057 7530 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:34.135149 7530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:34.135465 7530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:34.138351 7530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "2f49917d5259455c95d87b2d44bcd254"
format_stamp: "Formatted at 2026-08-11 12:51:34 on dist-test-slave-f0tn"
server_key: "f611487d06f35dcc5debafbb714f8fd1"
server_key_iv: "56d406c2a1e8da16985ba5d27c2a4094"
server_key_version: "encryptionkey@0"
I20260811 12:51:34.139089 7530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "2f49917d5259455c95d87b2d44bcd254"
format_stamp: "Formatted at 2026-08-11 12:51:34 on dist-test-slave-f0tn"
server_key: "f611487d06f35dcc5debafbb714f8fd1"
server_key_iv: "56d406c2a1e8da16985ba5d27c2a4094"
server_key_version: "encryptionkey@0"
I20260811 12:51:34.144202 7530 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 12:51:34.147639 7557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:34.148944 7530 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:51:34.149113 7530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "2f49917d5259455c95d87b2d44bcd254"
format_stamp: "Formatted at 2026-08-11 12:51:34 on dist-test-slave-f0tn"
server_key: "f611487d06f35dcc5debafbb714f8fd1"
server_key_iv: "56d406c2a1e8da16985ba5d27c2a4094"
server_key_version: "encryptionkey@0"
I20260811 12:51:34.149309 7530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:34.162573 7530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:34.163357 7530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:34.163622 7530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:34.164528 7530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:51:34.165993 7530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:51:34.166090 7530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:34.166200 7530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:51:34.166270 7530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:34.196959 7530 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:41845
I20260811 12:51:34.197000 7754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:41845 every 8 connection(s)
I20260811 12:51:34.198312 7530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:51:34.208539 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 7530
I20260811 12:51:34.208739 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:51:34.209300 7613 external_mini_cluster.cc:1428] Setting key dc3b62572cd977e677c185915b65a5fb
I20260811 12:51:34.210342 7318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.200808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:49485 (local address 127.7.111.126:40949)
0811 12:51:34.201014 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:34.201019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:34.202407 (+ 1388us) server_negotiation.cc:407] Connection header received
0811 12:51:34.204131 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:34.204135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:34.204205 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:34.204288 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:34.205242 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.206012 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.207528 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.207927 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.208020 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:34.209550 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:34.209555 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:34.209558 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:34.209577 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:34.209609 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:34.209620 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:34.209699 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:34.209955 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 12:51:34.210064 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:51:34.211499 7760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.201355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40949 (local address 127.7.111.67:49485)
0811 12:51:34.202259 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:34.202294 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:34.203893 (+ 1599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:34.204470 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:34.204477 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:34.204557 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:51:34.205109 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:34.205125 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.206131 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.206134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:34.207385 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:34.207401 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.208118 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.208125 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:34.208415 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:34.209130 (+ 715us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:51:34.209134 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:51:34.209185 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:51:34.209281 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:51:34.209724 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:51:34.209737 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:51:34.209745 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:51:34.209906 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 12:51:34.210251 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":282,"threads_started":1}
I20260811 12:51:34.212797 7755 heartbeater.cc:347] Connected to a master server at 127.7.111.126:40949
I20260811 12:51:34.213107 7755 heartbeater.cc:464] Registering TS with master...
I20260811 12:51:34.213981 7755 heartbeater.cc:511] Master 127.7.111.126:40949 requested a full tablet report, sending...
I20260811 12:51:34.215322 7009 ts_manager.cc:194] Registered new tserver with Master: 2f49917d5259455c95d87b2d44bcd254 (127.7.111.67:41845)
I20260811 12:51:34.216107 7009 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.111.67:49485
I20260811 12:51:34.224607 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:51:34.235351 7318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.228787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58524 (local address 127.7.111.126:40949)
0811 12:51:34.228980 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:34.228986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:34.229500 (+ 514us) server_negotiation.cc:407] Connection header received
0811 12:51:34.229670 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:34.229674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:34.229758 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:34.229835 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:34.230479 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.231275 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.233194 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.233673 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.233768 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:34.234799 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:34.234803 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:34.234807 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:34.234824 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:34.234855 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:34.234866 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:34.234970 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:34.235082 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 12:51:34.235127 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:51:34.239260 7009 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58524:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:51:34.246343 7009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:51:34.256418 7316 heartbeater.cc:503] Master 127.7.111.126:40949 was elected leader, sending a full tablet report...
I20260811 12:51:34.270817 7774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.258624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:41845 (local address 127.0.0.1:35720)
0811 12:51:34.259345 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:34.259398 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:34.259574 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:34.260631 (+ 1057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:34.260639 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:34.260671 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:34.261575 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:34.261592 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.264013 (+ 2421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.264021 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:34.266884 (+ 2863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:34.266906 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.269642 (+ 2736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.269651 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:34.269903 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:34.269968 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 12:51:34.270142 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 12:51:34.270379 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":322,"threads_started":1}
I20260811 12:51:34.271844 7776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.258828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35720 (local address 127.7.111.67:41845)
0811 12:51:34.259600 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:34.259610 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:34.259651 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:51:34.259769 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:34.259778 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:34.260071 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:34.260285 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:34.261888 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.263788 (+ 1900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.267169 (+ 3381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.268919 (+ 1750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.269093 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:34.269470 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:34.271157 (+ 1687us) server_negotiation.cc:299] Negotiation successful
0811 12:51:34.271390 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":1148,"threads_started":1}
I20260811 12:51:34.281948 7654 tablet_service.cc:1467] Processing CreateTablet for tablet e090088193994d20801ee7643998f024 (DEFAULT_TABLE table=test-table [id=164f4e2696d449e59dc1c62946e67f55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:51:34.283471 7774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.261197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:45045 (local address 127.0.0.1:58100)
0811 12:51:34.272262 (+ 11065us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:34.272287 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:34.272424 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:34.277601 (+ 5177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:34.277612 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:34.277640 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:34.277860 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:34.277873 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.279803 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.279809 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:34.281172 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:34.281190 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.282851 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.282856 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:34.283015 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:34.283044 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 12:51:34.283106 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 12:51:34.283217 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10908,"thread_start_us":16337,"threads_started":1}
I20260811 12:51:34.283610 7654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e090088193994d20801ee7643998f024. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:34.284543 7785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.261592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58100 (local address 127.7.111.65:45045)
0811 12:51:34.276778 (+ 15186us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:34.276788 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:34.276829 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:51:34.276972 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:34.276981 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:34.277283 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:34.277605 (+ 322us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:34.278445 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.279640 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.281423 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.282648 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.283753 (+ 1105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:34.284054 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:34.284116 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 12:51:34.284247 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14964,"thread_start_us":16225,"threads_started":1}
I20260811 12:51:34.286339 7781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.259599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:37337 (local address 127.0.0.1:60282)
0811 12:51:34.260154 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:34.260170 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:34.260295 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:34.278120 (+ 17825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:34.278126 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:34.278155 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:34.278550 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:34.278565 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.281760 (+ 3195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.281765 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:34.283713 (+ 1948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:34.283729 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.285575 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.285581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:34.285774 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:34.285807 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:51:34.285872 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 12:51:34.285994 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":1,"thread_start_us":587,"threads_started":1}
I20260811 12:51:34.287372 7782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.260201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60282 (local address 127.7.111.66:37337)
0811 12:51:34.277040 (+ 16839us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:34.277048 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:34.277079 (+ 31us) server_negotiation.cc:407] Connection header received
0811 12:51:34.277172 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:34.277179 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:34.277360 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:34.278137 (+ 777us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:34.278726 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.281602 (+ 2876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.283936 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.285329 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.286568 (+ 1239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:34.286815 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:34.286882 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 12:51:34.287040 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16663,"thread_start_us":16528,"threads_started":1}
I20260811 12:51:34.288007 7205 tablet_service.cc:1467] Processing CreateTablet for tablet e090088193994d20801ee7643998f024 (DEFAULT_TABLE table=test-table [id=164f4e2696d449e59dc1c62946e67f55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:51:34.289605 7205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e090088193994d20801ee7643998f024. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:34.291713 7414 tablet_service.cc:1467] Processing CreateTablet for tablet e090088193994d20801ee7643998f024 (DEFAULT_TABLE table=test-table [id=164f4e2696d449e59dc1c62946e67f55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:51:34.293118 7414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e090088193994d20801ee7643998f024. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:34.295877 7788 tablet_bootstrap.cc:492] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254: Bootstrap starting.
I20260811 12:51:34.299526 7788 tablet_bootstrap.cc:654] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:34.300856 7788 log.cc:824] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:34.303665 7791 tablet_bootstrap.cc:492] T e090088193994d20801ee7643998f024 P f2d7d98800684ca4bf9caa6c899e8954: Bootstrap starting.
I20260811 12:51:34.304248 7788 tablet_bootstrap.cc:492] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254: No bootstrap required, opened a new log
I20260811 12:51:34.304623 7788 ts_tablet_manager.cc:1397] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260811 12:51:34.307026 7791 tablet_bootstrap.cc:654] T e090088193994d20801ee7643998f024 P f2d7d98800684ca4bf9caa6c899e8954: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:34.308847 7792 tablet_bootstrap.cc:492] T e090088193994d20801ee7643998f024 P d4c1ffdce6e74d72a35ff2d729fcf57a: Bootstrap starting.
I20260811 12:51:34.309806 7791 log.cc:824] T e090088193994d20801ee7643998f024 P f2d7d98800684ca4bf9caa6c899e8954: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:34.312822 7792 tablet_bootstrap.cc:654] T e090088193994d20801ee7643998f024 P d4c1ffdce6e74d72a35ff2d729fcf57a: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:34.313524 7791 tablet_bootstrap.cc:492] T e090088193994d20801ee7643998f024 P f2d7d98800684ca4bf9caa6c899e8954: No bootstrap required, opened a new log
I20260811 12:51:34.313845 7791 ts_tablet_manager.cc:1397] T e090088193994d20801ee7643998f024 P f2d7d98800684ca4bf9caa6c899e8954: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260811 12:51:34.314308 7792 log.cc:824] T e090088193994d20801ee7643998f024 P d4c1ffdce6e74d72a35ff2d729fcf57a: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:34.314205 7788 raft_consensus.cc:348] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f49917d5259455c95d87b2d44bcd254" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41845 } } peers { permanent_uuid: "d4c1ffdce6e74d72a35ff2d729fcf57a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37337 } } peers { permanent_uuid: "f2d7d98800684ca4bf9caa6c899e8954" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 45045 } }
I20260811 12:51:34.314596 7788 raft_consensus.cc:374] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:34.314708 7788 raft_consensus.cc:729] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f49917d5259455c95d87b2d44bcd254, State: Initialized, Role: FOLLOWER
I20260811 12:51:34.315387 7788 consensus_queue.cc:260] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f49917d5259455c95d87b2d44bcd254" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41845 } } peers { permanent_uuid: "d4c1ffdce6e74d72a35ff2d729fcf57a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37337 } } peers { permanent_uuid: "f2d7d98800684ca4bf9caa6c899e8954" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 45045 } }
I20260811 12:51:34.316533 7755 heartbeater.cc:503] Master 127.7.111.126:40949 was elected leader, sending a full tablet report...
I20260811 12:51:34.317203 7788 ts_tablet_manager.cc:1428] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260811 12:51:34.317669 7792 tablet_bootstrap.cc:492] T e090088193994d20801ee7643998f024 P d4c1ffdce6e74d72a35ff2d729fcf57a: No bootstrap required, opened a new log
I20260811 12:51:34.317973 7792 ts_tablet_manager.cc:1397] T e090088193994d20801ee7643998f024 P d4c1ffdce6e74d72a35ff2d729fcf57a: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 12:51:34.326808 7791 raft_consensus.cc:348] T e090088193994d20801ee7643998f024 P f2d7d98800684ca4bf9caa6c899e8954 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f49917d5259455c95d87b2d44bcd254" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41845 } } peers { permanent_uuid: "d4c1ffdce6e74d72a35ff2d729fcf57a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37337 } } peers { permanent_uuid: "f2d7d98800684ca4bf9caa6c899e8954" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 45045 } }
I20260811 12:51:34.327224 7791 raft_consensus.cc:374] T e090088193994d20801ee7643998f024 P f2d7d98800684ca4bf9caa6c899e8954 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:34.327322 7791 raft_consensus.cc:729] T e090088193994d20801ee7643998f024 P f2d7d98800684ca4bf9caa6c899e8954 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2d7d98800684ca4bf9caa6c899e8954, State: Initialized, Role: FOLLOWER
I20260811 12:51:34.327068 7792 raft_consensus.cc:348] T e090088193994d20801ee7643998f024 P d4c1ffdce6e74d72a35ff2d729fcf57a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f49917d5259455c95d87b2d44bcd254" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41845 } } peers { permanent_uuid: "d4c1ffdce6e74d72a35ff2d729fcf57a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37337 } } peers { permanent_uuid: "f2d7d98800684ca4bf9caa6c899e8954" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 45045 } }
I20260811 12:51:34.327488 7792 raft_consensus.cc:374] T e090088193994d20801ee7643998f024 P d4c1ffdce6e74d72a35ff2d729fcf57a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:34.327589 7792 raft_consensus.cc:729] T e090088193994d20801ee7643998f024 P d4c1ffdce6e74d72a35ff2d729fcf57a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4c1ffdce6e74d72a35ff2d729fcf57a, State: Initialized, Role: FOLLOWER
I20260811 12:51:34.327920 7791 consensus_queue.cc:260] T e090088193994d20801ee7643998f024 P f2d7d98800684ca4bf9caa6c899e8954 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f49917d5259455c95d87b2d44bcd254" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41845 } } peers { permanent_uuid: "d4c1ffdce6e74d72a35ff2d729fcf57a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37337 } } peers { permanent_uuid: "f2d7d98800684ca4bf9caa6c899e8954" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 45045 } }
I20260811 12:51:34.328200 7792 consensus_queue.cc:260] T e090088193994d20801ee7643998f024 P d4c1ffdce6e74d72a35ff2d729fcf57a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f49917d5259455c95d87b2d44bcd254" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41845 } } peers { permanent_uuid: "d4c1ffdce6e74d72a35ff2d729fcf57a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37337 } } peers { permanent_uuid: "f2d7d98800684ca4bf9caa6c899e8954" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 45045 } }
I20260811 12:51:34.329329 7525 heartbeater.cc:503] Master 127.7.111.126:40949 was elected leader, sending a full tablet report...
I20260811 12:51:34.329775 7791 ts_tablet_manager.cc:1428] T e090088193994d20801ee7643998f024 P f2d7d98800684ca4bf9caa6c899e8954: Time spent starting tablet: real 0.016s user 0.009s sys 0.003s
I20260811 12:51:34.330088 7792 ts_tablet_manager.cc:1428] T e090088193994d20801ee7643998f024 P d4c1ffdce6e74d72a35ff2d729fcf57a: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260811 12:51:34.412923 7796 raft_consensus.cc:482] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:51:34.413223 7796 raft_consensus.cc:504] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f49917d5259455c95d87b2d44bcd254" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41845 } } peers { permanent_uuid: "d4c1ffdce6e74d72a35ff2d729fcf57a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37337 } } peers { permanent_uuid: "f2d7d98800684ca4bf9caa6c899e8954" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 45045 } }
I20260811 12:51:34.415078 7796 leader_election.cc:288] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4c1ffdce6e74d72a35ff2d729fcf57a (127.7.111.66:37337), f2d7d98800684ca4bf9caa6c899e8954 (127.7.111.65:45045)
I20260811 12:51:34.421175 7782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.415416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:57163 (local address 127.7.111.66:37337)
0811 12:51:34.415655 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:34.415662 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:34.415688 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:51:34.415900 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:34.415906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:34.415992 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:34.416070 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:34.416562 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.417503 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.419332 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.420296 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.420405 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:34.420454 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:34.420839 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 12:51:34.420929 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 12:51:34.421270 7760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.415302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:37337 (local address 127.7.111.67:57163)
0811 12:51:34.415560 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:34.415582 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:34.415701 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:34.416176 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:34.416181 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:34.416215 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:34.416399 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:34.416411 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.417638 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.417644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:34.419180 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:34.419193 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.420449 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.420455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:34.420656 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:34.420704 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 12:51:34.420814 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 12:51:34.420948 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143}
I20260811 12:51:34.421800 7801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.415384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:45045 (local address 127.7.111.67:50333)
0811 12:51:34.416211 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:34.416225 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:34.416334 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:34.416662 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:34.416667 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:34.416692 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:34.417054 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:34.417066 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.418219 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.418223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:34.419741 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:34.419755 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.421109 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.421115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:34.421302 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:34.421333 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:51:34.421455 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 12:51:34.421572 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"thread_start_us":501,"threads_started":1}
I20260811 12:51:34.422248 7785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.415689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:50333 (local address 127.7.111.65:45045)
0811 12:51:34.415894 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:34.415899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:34.416313 (+ 414us) server_negotiation.cc:407] Connection header received
0811 12:51:34.416451 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:34.416456 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:34.416544 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:34.416621 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:34.417209 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.418071 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.419924 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.420943 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.421097 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:34.421163 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:34.421885 (+ 722us) server_negotiation.cc:299] Negotiation successful
0811 12:51:34.422014 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 12:51:34.422411 7450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e090088193994d20801ee7643998f024" candidate_uuid: "2f49917d5259455c95d87b2d44bcd254" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4c1ffdce6e74d72a35ff2d729fcf57a" is_pre_election: true
I20260811 12:51:34.423062 7450 raft_consensus.cc:2454] T e090088193994d20801ee7643998f024 P d4c1ffdce6e74d72a35ff2d729fcf57a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f49917d5259455c95d87b2d44bcd254 in term 0.
I20260811 12:51:34.423089 7241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e090088193994d20801ee7643998f024" candidate_uuid: "2f49917d5259455c95d87b2d44bcd254" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2d7d98800684ca4bf9caa6c899e8954" is_pre_election: true
I20260811 12:51:34.423597 7241 raft_consensus.cc:2454] T e090088193994d20801ee7643998f024 P f2d7d98800684ca4bf9caa6c899e8954 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f49917d5259455c95d87b2d44bcd254 in term 0.
I20260811 12:51:34.423686 7559 leader_election.cc:302] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f49917d5259455c95d87b2d44bcd254, d4c1ffdce6e74d72a35ff2d729fcf57a; no voters:
I20260811 12:51:34.424093 7796 raft_consensus.cc:2781] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:51:34.424253 7796 raft_consensus.cc:482] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:51:34.424366 7796 raft_consensus.cc:3037] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:34.426723 7796 raft_consensus.cc:504] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f49917d5259455c95d87b2d44bcd254" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41845 } } peers { permanent_uuid: "d4c1ffdce6e74d72a35ff2d729fcf57a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37337 } } peers { permanent_uuid: "f2d7d98800684ca4bf9caa6c899e8954" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 45045 } }
I20260811 12:51:34.427284 7796 leader_election.cc:288] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254 [CANDIDATE]: Term 1 election: Requested vote from peers d4c1ffdce6e74d72a35ff2d729fcf57a (127.7.111.66:37337), f2d7d98800684ca4bf9caa6c899e8954 (127.7.111.65:45045)
I20260811 12:51:34.427721 7450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e090088193994d20801ee7643998f024" candidate_uuid: "2f49917d5259455c95d87b2d44bcd254" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4c1ffdce6e74d72a35ff2d729fcf57a"
I20260811 12:51:34.427778 7241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e090088193994d20801ee7643998f024" candidate_uuid: "2f49917d5259455c95d87b2d44bcd254" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2d7d98800684ca4bf9caa6c899e8954"
I20260811 12:51:34.427932 7450 raft_consensus.cc:3037] T e090088193994d20801ee7643998f024 P d4c1ffdce6e74d72a35ff2d729fcf57a [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:34.427948 7241 raft_consensus.cc:3037] T e090088193994d20801ee7643998f024 P f2d7d98800684ca4bf9caa6c899e8954 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:34.431115 7241 raft_consensus.cc:2454] T e090088193994d20801ee7643998f024 P f2d7d98800684ca4bf9caa6c899e8954 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f49917d5259455c95d87b2d44bcd254 in term 1.
I20260811 12:51:34.431119 7450 raft_consensus.cc:2454] T e090088193994d20801ee7643998f024 P d4c1ffdce6e74d72a35ff2d729fcf57a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f49917d5259455c95d87b2d44bcd254 in term 1.
I20260811 12:51:34.431545 7559 leader_election.cc:302] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f49917d5259455c95d87b2d44bcd254, d4c1ffdce6e74d72a35ff2d729fcf57a; no voters:
I20260811 12:51:34.431862 7796 raft_consensus.cc:2781] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:51:34.432303 7796 raft_consensus.cc:686] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254 [term 1 LEADER]: Becoming Leader. State: Replica: 2f49917d5259455c95d87b2d44bcd254, State: Running, Role: LEADER
I20260811 12:51:34.432835 7796 consensus_queue.cc:237] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f49917d5259455c95d87b2d44bcd254" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41845 } } peers { permanent_uuid: "d4c1ffdce6e74d72a35ff2d729fcf57a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37337 } } peers { permanent_uuid: "f2d7d98800684ca4bf9caa6c899e8954" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 45045 } }
I20260811 12:51:34.437489 7009 catalog_manager.cc:5495] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f49917d5259455c95d87b2d44bcd254 (127.7.111.67). New cstate: current_term: 1 leader_uuid: "2f49917d5259455c95d87b2d44bcd254" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f49917d5259455c95d87b2d44bcd254" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 41845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4c1ffdce6e74d72a35ff2d729fcf57a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2d7d98800684ca4bf9caa6c899e8954" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 45045 } health_report { overall_health: UNKNOWN } } }
W20260811 12:51:34.449934 7756 tablet.cc:2369] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:51:34.457707 7527 tablet.cc:2369] T e090088193994d20801ee7643998f024 P d4c1ffdce6e74d72a35ff2d729fcf57a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:51:34.461294 7776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.456018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35736 (local address 127.7.111.67:41845)
0811 12:51:34.456179 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:34.456185 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:34.456333 (+ 148us) server_negotiation.cc:407] Connection header received
0811 12:51:34.456498 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:34.456501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:34.456575 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:34.456708 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:51:34.457006 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.457749 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.459146 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.459565 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.459683 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:34.460269 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:51:34.460746 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:51:34.460913 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:34.461047 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 12:51:34.461087 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:51:34.469040 7241 raft_consensus.cc:1261] T e090088193994d20801ee7643998f024 P f2d7d98800684ca4bf9caa6c899e8954 [term 1 FOLLOWER]: Refusing update from remote peer 2f49917d5259455c95d87b2d44bcd254: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:51:34.469044 7450 raft_consensus.cc:1261] T e090088193994d20801ee7643998f024 P d4c1ffdce6e74d72a35ff2d729fcf57a [term 1 FOLLOWER]: Refusing update from remote peer 2f49917d5259455c95d87b2d44bcd254: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:51:34.469944 7802 consensus_queue.cc:1035] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d7d98800684ca4bf9caa6c899e8954" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:51:34.470296 7796 consensus_queue.cc:1035] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4c1ffdce6e74d72a35ff2d729fcf57a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:51:34.500758 7009 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:51:34.501053 7009 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:51:34.504901 7009 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18b1a008752b434aa4ede1c8f816a996: Sending DeleteTablet for 3 replicas of tablet e090088193994d20801ee7643998f024
I20260811 12:51:34.505998 7654 tablet_service.cc:1514] Processing DeleteTablet for tablet e090088193994d20801ee7643998f024 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:51:34 UTC) from {username='slave'} at 127.0.0.1:35720
I20260811 12:51:34.506171 7414 tablet_service.cc:1514] Processing DeleteTablet for tablet e090088193994d20801ee7643998f024 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:51:34 UTC) from {username='slave'} at 127.0.0.1:60282
I20260811 12:51:34.506426 7205 tablet_service.cc:1514] Processing DeleteTablet for tablet e090088193994d20801ee7643998f024 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:51:34 UTC) from {username='slave'} at 127.0.0.1:58100
I20260811 12:51:34.508515 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 7105
I20260811 12:51:34.524674 7828 tablet_replica.cc:330] T e090088193994d20801ee7643998f024 P d4c1ffdce6e74d72a35ff2d729fcf57a: stopping tablet replica
I20260811 12:51:34.525874 7828 raft_consensus.cc:2229] T e090088193994d20801ee7643998f024 P d4c1ffdce6e74d72a35ff2d729fcf57a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:51:34.526288 7831 tablet_replica.cc:330] T e090088193994d20801ee7643998f024 P f2d7d98800684ca4bf9caa6c899e8954: stopping tablet replica
I20260811 12:51:34.526700 7828 raft_consensus.cc:2258] T e090088193994d20801ee7643998f024 P d4c1ffdce6e74d72a35ff2d729fcf57a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:51:34.526767 7831 raft_consensus.cc:2229] T e090088193994d20801ee7643998f024 P f2d7d98800684ca4bf9caa6c899e8954 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:51:34.527319 7831 raft_consensus.cc:2258] T e090088193994d20801ee7643998f024 P f2d7d98800684ca4bf9caa6c899e8954 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:51:34.529273 7828 ts_tablet_manager.cc:1905] T e090088193994d20801ee7643998f024 P d4c1ffdce6e74d72a35ff2d729fcf57a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:51:34.531219 7831 ts_tablet_manager.cc:1905] T e090088193994d20801ee7643998f024 P f2d7d98800684ca4bf9caa6c899e8954: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:51:34.531876 7827 tablet_replica.cc:330] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254: stopping tablet replica
I20260811 12:51:34.532301 7827 raft_consensus.cc:2229] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:51:34.533078 7827 raft_consensus.cc:2258] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:51:34.534996 7827 ts_tablet_manager.cc:1905] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:51:34.535815 7828 ts_tablet_manager.cc:1918] T e090088193994d20801ee7643998f024 P d4c1ffdce6e74d72a35ff2d729fcf57a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:51:34.536034 7828 log.cc:1196] T e090088193994d20801ee7643998f024 P d4c1ffdce6e74d72a35ff2d729fcf57a: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/e090088193994d20801ee7643998f024
I20260811 12:51:34.536688 7828 ts_tablet_manager.cc:1939] T e090088193994d20801ee7643998f024 P d4c1ffdce6e74d72a35ff2d729fcf57a: Deleting consensus metadata
I20260811 12:51:34.538607 6977 catalog_manager.cc:4841] TS d4c1ffdce6e74d72a35ff2d729fcf57a (127.7.111.66:37337): tablet e090088193994d20801ee7643998f024 (table test-table [id=164f4e2696d449e59dc1c62946e67f55]) successfully deleted
I20260811 12:51:34.541147 7785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.509976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58108 (local address 127.7.111.65:45045)
0811 12:51:34.510164 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:34.510171 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:34.514789 (+ 4618us) server_negotiation.cc:407] Connection header received
0811 12:51:34.521085 (+ 6296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:34.521092 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:34.521220 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:34.521313 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:34.522583 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.523772 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.527770 (+ 3998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.528413 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.528516 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:34.535943 (+ 7427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:34.535955 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:34.535992 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:34.536753 (+ 761us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:34.536814 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:34.536841 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:34.536942 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:34.537264 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 12:51:34.540768 (+ 3504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:51:34.542601 7307 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:34.543486 7827 ts_tablet_manager.cc:1918] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:51:34.543725 7827 log.cc:1196] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/e090088193994d20801ee7643998f024
I20260811 12:51:34.544341 7827 ts_tablet_manager.cc:1939] T e090088193994d20801ee7643998f024 P 2f49917d5259455c95d87b2d44bcd254: Deleting consensus metadata
I20260811 12:51:34.546236 6978 catalog_manager.cc:4841] TS 2f49917d5259455c95d87b2d44bcd254 (127.7.111.67:41845): tablet e090088193994d20801ee7643998f024 (table test-table [id=164f4e2696d449e59dc1c62946e67f55]) successfully deleted
I20260811 12:51:34.630535 7831 ts_tablet_manager.cc:1918] T e090088193994d20801ee7643998f024 P f2d7d98800684ca4bf9caa6c899e8954: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:51:34.630790 7831 log.cc:1196] T e090088193994d20801ee7643998f024 P f2d7d98800684ca4bf9caa6c899e8954: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/e090088193994d20801ee7643998f024
I20260811 12:51:34.630828 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 7105
I20260811 12:51:34.666538 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 7320
W20260811 12:51:34.666563 6978 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.111.65:45045 (error 108)
I20260811 12:51:34.669484 7781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.668921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:45045 (local address 127.0.0.1:59608)
0811 12:51:34.669130 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:34.669270 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260811 12:51:34.669782 6978 catalog_manager.cc:4584] TS f2d7d98800684ca4bf9caa6c899e8954 (127.7.111.65:45045): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e090088193994d20801ee7643998f024: Network error: Client connection negotiation failed: client connection to 127.7.111.65:45045: connect: Connection refused (error 111)
I20260811 12:51:34.675078 7782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.667443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44236 (local address 127.7.111.66:37337)
0811 12:51:34.667646 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:34.667652 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:34.667771 (+ 119us) server_negotiation.cc:407] Connection header received
0811 12:51:34.667956 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:34.667962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:34.668076 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:34.668201 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:34.668872 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.669763 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.671524 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.672169 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.672292 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:34.673456 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:34.673469 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:34.673489 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:34.674273 (+ 784us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:34.674354 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:34.674380 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:34.674501 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:34.674633 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 12:51:34.674745 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:51:34.675779 7519 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:34.723315 7781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.722837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:45045 (local address 127.0.0.1:59612)
0811 12:51:34.723046 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:34.723136 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 12:51:34.724573 7781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.724145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:45045 (local address 127.0.0.1:59624)
0811 12:51:34.724325 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:34.724421 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 12:51:34.782639 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 7320
I20260811 12:51:34.787155 7781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.786678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:45045 (local address 127.0.0.1:59640)
0811 12:51:34.786880 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:34.786971 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 12:51:34.790227 7781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.789866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:45045 (local address 127.0.0.1:59648)
0811 12:51:34.790028 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:34.790108 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 12:51:34.813905 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 7530
I20260811 12:51:34.821056 7776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.814793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32964 (local address 127.7.111.67:41845)
0811 12:51:34.814999 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:34.815004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:34.815025 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:51:34.815142 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:34.815147 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:34.815255 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:34.815341 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:34.815798 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.816606 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.818034 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:34.818499 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:34.818612 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:34.819712 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:34.819720 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:34.819739 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:34.820317 (+ 578us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:34.820366 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:34.820385 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:34.820476 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:34.820640 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 12:51:34.820735 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:51:34.821767 7750 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:34.871049 7781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.870520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:45045 (local address 127.0.0.1:59658)
0811 12:51:34.870742 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:34.870885 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 12:51:34.872108 7781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:34.871733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:45045 (local address 127.0.0.1:59664)
0811 12:51:34.871915 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:34.871988 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260811 12:51:34.942999 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 7530
I20260811 12:51:34.973795 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 6949
I20260811 12:51:34.974915 7058 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:35.082666 7781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:35.082052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:45045 (local address 127.0.0.1:59668)
0811 12:51:35.082256 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:35.082364 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.65:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 12:51:35.083103 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 6949
2026-08-11T12:51:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2938 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:51:35 dist-test-slave-f0tn krb5kdc[7848](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:51:35 dist-test-slave-f0tn krb5kdc[7848](info): set up 2 sockets
Aug 11 12:51:35 dist-test-slave-f0tn krb5kdc[7848](info): commencing operation
krb5kdc: starting...
W20260811 12:51:36.357894 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:51:36 dist-test-slave-f0tn krb5kdc[7848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:51:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:51:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:51:36.699774 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:41559
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:44293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:41559
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:51:37.058672 7870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:37.058943 7870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:37.059046 7870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:37.066092 7870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:51:37.066195 7870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:37.066247 7870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:37.066303 7870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:51:37.066345 7870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:51:37.075829 7870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:44293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:41559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:41559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:37.077070 7870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:37.078367 7870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:37.090238 7879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:37.090188 7882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:37.090262 7880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:37.090690 7870 server_base.cc:1034] running on GCE node
I20260811 12:51:37.091557 7870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:37.092877 7870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:37.094079 7870 hybrid_clock.cc:648] HybridClock initialized: now 1786452697094023 us; error 33 us; skew 500 ppm
I20260811 12:51:37.094448 7870 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:51:37 dist-test-slave-f0tn krb5kdc[7848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452697, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:51:37.097440 7870 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:51:37.098860 7870 webserver.cc:460] Webserver started at http://127.7.111.126:37191/ using document root <none> and password file <none>
I20260811 12:51:37.099516 7870 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:37.099637 7870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:37.099936 7870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:37.103031 7870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "6788004703114cbcbffb18842273ab65"
format_stamp: "Formatted at 2026-08-11 12:51:37 on dist-test-slave-f0tn"
server_key: "cb2e60a5f60f0a6b0aac4601609e53e5"
server_key_iv: "8ada6dad947e08e3be8a4b702d2b4856"
server_key_version: "encryptionkey@0"
I20260811 12:51:37.103765 7870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "6788004703114cbcbffb18842273ab65"
format_stamp: "Formatted at 2026-08-11 12:51:37 on dist-test-slave-f0tn"
server_key: "cb2e60a5f60f0a6b0aac4601609e53e5"
server_key_iv: "8ada6dad947e08e3be8a4b702d2b4856"
server_key_version: "encryptionkey@0"
I20260811 12:51:37.108795 7870 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:51:37.112239 7890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:37.113548 7870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:51:37.113710 7870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "6788004703114cbcbffb18842273ab65"
format_stamp: "Formatted at 2026-08-11 12:51:37 on dist-test-slave-f0tn"
server_key: "cb2e60a5f60f0a6b0aac4601609e53e5"
server_key_iv: "8ada6dad947e08e3be8a4b702d2b4856"
server_key_version: "encryptionkey@0"
I20260811 12:51:37.113881 7870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:37.134009 7870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:37.137883 7870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:37.138157 7870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:37.156512 7870 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:41559
I20260811 12:51:37.156546 7981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:41559 every 8 connection(s)
I20260811 12:51:37.157923 7870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:51:37.160588 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 7870
I20260811 12:51:37.160775 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:51:37.161322 7613 external_mini_cluster.cc:1428] Setting key e1044a8fdc25204120866c2b4ab479cf
I20260811 12:51:37.162406 7984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 12:51:37 dist-test-slave-f0tn krb5kdc[7848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:51:37.176498 7984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65: Bootstrap starting.
I20260811 12:51:37.179879 7984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:37.181156 7984 log.cc:824] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:37.184448 7984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65: No bootstrap required, opened a new log
I20260811 12:51:37.184705 7989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:37.163268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35818 (local address 127.7.111.126:41559)
0811 12:51:37.164062 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:37.164074 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:37.164124 (+ 50us) server_negotiation.cc:407] Connection header received
0811 12:51:37.165777 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:37.165800 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:37.166193 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:37.166523 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:37.168328 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:37.169733 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:37.171399 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:37.172057 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:37.172198 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:37.177169 (+ 4971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:37.177177 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:37.177193 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:37.177235 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:37.180306 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:37.181148 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:37.181155 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:37.181162 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:37.181358 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:37.181861 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:37.181870 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:37.181873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:37.182652 (+ 779us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:37.182868 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:37.183332 (+ 464us) server_negotiation.cc:299] Negotiation successful
0811 12:51:37.183673 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":356,"threads_started":1}
I20260811 12:51:37.192377 7984 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6788004703114cbcbffb18842273ab65" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 41559 } }
I20260811 12:51:37.192723 7984 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:37.192817 7984 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6788004703114cbcbffb18842273ab65, State: Initialized, Role: FOLLOWER
I20260811 12:51:37.193339 7984 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6788004703114cbcbffb18842273ab65" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 41559 } }
I20260811 12:51:37.193595 7984 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:51:37.193725 7984 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:51:37.193867 7984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:37.195915 7984 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6788004703114cbcbffb18842273ab65" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 41559 } }
I20260811 12:51:37.196408 7984 leader_election.cc:302] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6788004703114cbcbffb18842273ab65; no voters:
I20260811 12:51:37.196940 7984 leader_election.cc:288] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:51:37.197072 7992 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:51:37.198096 7992 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65 [term 1 LEADER]: Becoming Leader. State: Replica: 6788004703114cbcbffb18842273ab65, State: Running, Role: LEADER
I20260811 12:51:37.198659 7992 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6788004703114cbcbffb18842273ab65" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 41559 } }
I20260811 12:51:37.199098 7984 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:51:37.201653 7993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6788004703114cbcbffb18842273ab65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6788004703114cbcbffb18842273ab65" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 41559 } } }
I20260811 12:51:37.202035 7993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65 [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:37.202483 7994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6788004703114cbcbffb18842273ab65. Latest consensus state: current_term: 1 leader_uuid: "6788004703114cbcbffb18842273ab65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6788004703114cbcbffb18842273ab65" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 41559 } } }
I20260811 12:51:37.203156 7994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65 [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:37.204597 7999 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:51:37.209815 7999 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:51:37.219319 7999 catalog_manager.cc:1347] Generated new cluster ID: 5f6f6a93c29d4f50add0d2ae00196ef5
I20260811 12:51:37.219465 7999 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:51:37.246879 7999 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:51:37.248268 7999 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:51:37.254058 7999 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65: Generated new TSK 0
I20260811 12:51:37.255205 7999 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.111.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
Entry for principal kudu/127.7.111.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab.
I20260811 12:51:37.339802 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:41559
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:44293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:51:37.687671 8030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:37.687958 8030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:37.688078 8030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:37.695534 8030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:37.695648 8030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:37.695780 8030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:51:37.705783 8030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:44293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.65.keytab
--principal=kudu/127.7.111.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:41559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:37.707108 8030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:37.708491 8030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:37.721653 8044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:37.721760 8039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:37.721695 8040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:37.722676 8030 server_base.cc:1034] running on GCE node
I20260811 12:51:37.723300 8030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:37.724201 8030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:37.725436 8030 hybrid_clock.cc:648] HybridClock initialized: now 1786452697725355 us; error 35 us; skew 500 ppm
I20260811 12:51:37.725742 8030 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:51:37 dist-test-slave-f0tn krb5kdc[7848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452697, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:51:37.728652 8030 init.cc:377] Logged in from keytab as kudu/127.7.111.65@KRBTEST.COM (short username kudu)
I20260811 12:51:37.730090 8030 webserver.cc:460] Webserver started at http://127.7.111.65:34091/ using document root <none> and password file <none>
I20260811 12:51:37.730738 8030 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:37.730856 8030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:37.731153 8030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:37.734022 8030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "be1f4effe9b043869cccaac687eaf7b3"
format_stamp: "Formatted at 2026-08-11 12:51:37 on dist-test-slave-f0tn"
server_key: "1047a492f7f950a3ae18690aaad37f61"
server_key_iv: "3a7e0269581be731526b3decf26a0e3d"
server_key_version: "encryptionkey@0"
I20260811 12:51:37.734778 8030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "be1f4effe9b043869cccaac687eaf7b3"
format_stamp: "Formatted at 2026-08-11 12:51:37 on dist-test-slave-f0tn"
server_key: "1047a492f7f950a3ae18690aaad37f61"
server_key_iv: "3a7e0269581be731526b3decf26a0e3d"
server_key_version: "encryptionkey@0"
I20260811 12:51:37.739738 8030 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 12:51:37.743185 8055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:37.744442 8030 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:51:37.744590 8030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "be1f4effe9b043869cccaac687eaf7b3"
format_stamp: "Formatted at 2026-08-11 12:51:37 on dist-test-slave-f0tn"
server_key: "1047a492f7f950a3ae18690aaad37f61"
server_key_iv: "3a7e0269581be731526b3decf26a0e3d"
server_key_version: "encryptionkey@0"
I20260811 12:51:37.744784 8030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:37.765287 8030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:37.768985 8030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:37.769249 8030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:37.770207 8030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:51:37.771540 8030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:51:37.771622 8030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:37.771740 8030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:51:37.771795 8030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:37.802871 8030 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:36385
I20260811 12:51:37.802918 8241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:36385 every 8 connection(s)
I20260811 12:51:37.804131 8030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:51:37.814011 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 8030
I20260811 12:51:37.814194 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:51:37.814610 7613 external_mini_cluster.cc:1428] Setting key 3a6d8eb8ddd37a898432432080f9554b
Aug 11 12:51:37 dist-test-slave-f0tn krb5kdc[7848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452697, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.65@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:51:37.823366 8250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:37.806640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:40133 (local address 127.7.111.126:41559)
0811 12:51:37.807083 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:37.807088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:37.807922 (+ 834us) server_negotiation.cc:407] Connection header received
0811 12:51:37.809785 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:37.809791 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:37.809874 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:37.809971 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:37.811486 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:37.812225 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:37.813799 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:37.814210 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:37.814332 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:37.817949 (+ 3617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:37.817954 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:37.817957 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:37.817990 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:37.820489 (+ 2499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:37.821173 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:37.821179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:37.821183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:37.821306 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:37.821720 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:37.821723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:37.821725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:37.821985 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:37.822117 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:37.822780 (+ 663us) server_negotiation.cc:299] Negotiation successful
0811 12:51:37.822997 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":236,"threads_started":1}
I20260811 12:51:37.824242 8251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:37.807085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:41559 (local address 127.7.111.65:40133)
0811 12:51:37.807765 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:37.807803 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:37.809521 (+ 1718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:37.810223 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:37.810231 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:37.810778 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:51:37.811311 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:37.811331 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:37.812371 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:37.812375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:37.813638 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:37.813650 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:37.814471 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:37.814478 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:37.814720 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:37.815349 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:51:37.815372 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:51:37.817745 (+ 2373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:51:37.820614 (+ 2869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:51:37.820622 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:51:37.820637 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:51:37.821055 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:51:37.821426 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:51:37.821431 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:51:37.821434 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:51:37.821606 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:51:37.822108 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:51:37.822115 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:51:37.822420 (+ 305us) client_negotiation.cc:769] Sending connection context
0811 12:51:37.822643 (+ 223us) client_negotiation.cc:240] Negotiation successful
0811 12:51:37.823141 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":254,"threads_started":1}
I20260811 12:51:37.825762 8246 heartbeater.cc:347] Connected to a master server at 127.7.111.126:41559
I20260811 12:51:37.826051 8246 heartbeater.cc:464] Registering TS with master...
I20260811 12:51:37.826753 8246 heartbeater.cc:511] Master 127.7.111.126:41559 requested a full tablet report, sending...
I20260811 12:51:37.828913 7923 ts_manager.cc:194] Registered new tserver with Master: be1f4effe9b043869cccaac687eaf7b3 (127.7.111.65:36385)
I20260811 12:51:37.830742 7923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.65@KRBTEST.COM'} at 127.7.111.65:40133
WARNING: no policy specified for kudu/127.7.111.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
Entry for principal kudu/127.7.111.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab.
I20260811 12:51:37.881942 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:41559
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:44293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:51:38.221117 8254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:38.221426 8254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:38.221534 8254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:38.229421 8254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:38.229552 8254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:38.229676 8254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:51:38.239724 8254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:44293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.66.keytab
--principal=kudu/127.7.111.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:41559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:38.240983 8254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:38.242388 8254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:38.257370 8264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:38.257418 8269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:38.257639 8263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:38.258306 8254 server_base.cc:1034] running on GCE node
I20260811 12:51:38.258934 8254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:38.259878 8254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:38.261077 8254 hybrid_clock.cc:648] HybridClock initialized: now 1786452698261027 us; error 39 us; skew 500 ppm
I20260811 12:51:38.261404 8254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:51:38 dist-test-slave-f0tn krb5kdc[7848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452698, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:51:38.264443 8254 init.cc:377] Logged in from keytab as kudu/127.7.111.66@KRBTEST.COM (short username kudu)
I20260811 12:51:38.265903 8254 webserver.cc:460] Webserver started at http://127.7.111.66:36709/ using document root <none> and password file <none>
I20260811 12:51:38.266583 8254 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:38.266716 8254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:38.267023 8254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:38.270032 8254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "e737e8637baa4b5aa70c3031df23d62a"
format_stamp: "Formatted at 2026-08-11 12:51:38 on dist-test-slave-f0tn"
server_key: "85ee5aa1b0031a92d119a45350177c0e"
server_key_iv: "5f3d400260005471b69aac3165d314ef"
server_key_version: "encryptionkey@0"
I20260811 12:51:38.270839 8254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "e737e8637baa4b5aa70c3031df23d62a"
format_stamp: "Formatted at 2026-08-11 12:51:38 on dist-test-slave-f0tn"
server_key: "85ee5aa1b0031a92d119a45350177c0e"
server_key_iv: "5f3d400260005471b69aac3165d314ef"
server_key_version: "encryptionkey@0"
I20260811 12:51:38.275977 8254 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:51:38.279388 8284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:38.280637 8254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:51:38.280786 8254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "e737e8637baa4b5aa70c3031df23d62a"
format_stamp: "Formatted at 2026-08-11 12:51:38 on dist-test-slave-f0tn"
server_key: "85ee5aa1b0031a92d119a45350177c0e"
server_key_iv: "5f3d400260005471b69aac3165d314ef"
server_key_version: "encryptionkey@0"
I20260811 12:51:38.280977 8254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:38.315084 8254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:38.319527 8254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:38.319813 8254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:38.320837 8254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:51:38.322280 8254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:51:38.322361 8254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:38.322489 8254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:51:38.322542 8254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:38.353981 8254 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:37835
I20260811 12:51:38.354032 8475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:37835 every 8 connection(s)
I20260811 12:51:38.355283 8254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:51:38.362293 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 8254
I20260811 12:51:38.362466 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:51:38.362926 7613 external_mini_cluster.cc:1428] Setting key afc4708b9a2930b8fb338e797a3d5624
Aug 11 12:51:38 dist-test-slave-f0tn krb5kdc[7848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452698, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.66@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:51:38.376892 8483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:38.357821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:45659 (local address 127.7.111.126:41559)
0811 12:51:38.358271 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:38.358275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:38.359244 (+ 969us) server_negotiation.cc:407] Connection header received
0811 12:51:38.361503 (+ 2259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:38.361508 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:38.361595 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:38.361708 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:38.363394 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:38.364372 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:38.366617 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:38.367061 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:38.367159 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:38.371153 (+ 3994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:38.371159 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:38.371162 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:38.371194 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:38.373793 (+ 2599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:38.374621 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:38.374625 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:38.374628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:38.374768 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:38.375217 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:38.375220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:38.375222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:38.375492 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:38.375604 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:38.376417 (+ 813us) server_negotiation.cc:299] Negotiation successful
0811 12:51:38.376595 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":223,"threads_started":1}
I20260811 12:51:38.378325 8484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:38.358384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:41559 (local address 127.7.111.66:45659)
0811 12:51:38.359077 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:38.359123 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:38.361201 (+ 2078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:38.361999 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:38.362007 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:38.362577 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:51:38.363215 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:38.363236 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:38.364579 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:38.364583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:38.366464 (+ 1881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:38.366479 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:38.367302 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:38.367309 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:38.367603 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:38.368301 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:51:38.368337 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:51:38.370913 (+ 2576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:51:38.373926 (+ 3013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:51:38.373937 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:51:38.373956 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:51:38.374508 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:51:38.374866 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:51:38.374870 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:51:38.374873 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:51:38.375127 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:51:38.375606 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:51:38.375613 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:51:38.376051 (+ 438us) client_negotiation.cc:769] Sending connection context
0811 12:51:38.376353 (+ 302us) client_negotiation.cc:240] Negotiation successful
0811 12:51:38.376931 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":280,"threads_started":1}
I20260811 12:51:38.379892 8478 heartbeater.cc:347] Connected to a master server at 127.7.111.126:41559
I20260811 12:51:38.380205 8478 heartbeater.cc:464] Registering TS with master...
I20260811 12:51:38.380986 8478 heartbeater.cc:511] Master 127.7.111.126:41559 requested a full tablet report, sending...
I20260811 12:51:38.382727 7923 ts_manager.cc:194] Registered new tserver with Master: e737e8637baa4b5aa70c3031df23d62a (127.7.111.66:37835)
I20260811 12:51:38.383554 7923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.66@KRBTEST.COM'} at 127.7.111.66:45659
WARNING: no policy specified for kudu/127.7.111.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
Entry for principal kudu/127.7.111.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab.
I20260811 12:51:38.438582 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:41559
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:44293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:51:38.780879 8487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:38.781148 8487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:38.781250 8487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:38.788599 8487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:38.788707 8487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:38.788825 8487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:51:38.798261 8487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:44293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.67.keytab
--principal=kudu/127.7.111.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:41559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:38.799469 8487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:38.800815 8487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:38.814000 8497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:38.814181 8500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:38.814108 8496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:38.815081 8487 server_base.cc:1034] running on GCE node
I20260811 12:51:38.815630 8487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:38.816505 8487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:38.817711 8487 hybrid_clock.cc:648] HybridClock initialized: now 1786452698817646 us; error 46 us; skew 500 ppm
I20260811 12:51:38.818001 8487 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:51:38 dist-test-slave-f0tn krb5kdc[7848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452698, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:51:38.821120 8487 init.cc:377] Logged in from keytab as kudu/127.7.111.67@KRBTEST.COM (short username kudu)
I20260811 12:51:38.822508 8487 webserver.cc:460] Webserver started at http://127.7.111.67:42803/ using document root <none> and password file <none>
I20260811 12:51:38.823155 8487 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:38.823273 8487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:38.823573 8487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:38.826403 8487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "4a980788662940379ffb7cc883ad2a5b"
format_stamp: "Formatted at 2026-08-11 12:51:38 on dist-test-slave-f0tn"
server_key: "2774b6a843db30b5062d16fc6f7cd181"
server_key_iv: "09fb46b0ff2bb03ba225e60b58a23c20"
server_key_version: "encryptionkey@0"
I20260811 12:51:38.827153 8487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "4a980788662940379ffb7cc883ad2a5b"
format_stamp: "Formatted at 2026-08-11 12:51:38 on dist-test-slave-f0tn"
server_key: "2774b6a843db30b5062d16fc6f7cd181"
server_key_iv: "09fb46b0ff2bb03ba225e60b58a23c20"
server_key_version: "encryptionkey@0"
I20260811 12:51:38.831991 8487 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 12:51:38.833585 8246 heartbeater.cc:503] Master 127.7.111.126:41559 was elected leader, sending a full tablet report...
I20260811 12:51:38.835490 8512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:38.836712 8487 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:51:38.836853 8487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "4a980788662940379ffb7cc883ad2a5b"
format_stamp: "Formatted at 2026-08-11 12:51:38 on dist-test-slave-f0tn"
server_key: "2774b6a843db30b5062d16fc6f7cd181"
server_key_iv: "09fb46b0ff2bb03ba225e60b58a23c20"
server_key_version: "encryptionkey@0"
I20260811 12:51:38.837029 8487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:38.850792 8487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:38.854605 8487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:38.854862 8487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:38.855736 8487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:51:38.857004 8487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:51:38.857076 8487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:38.857184 8487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:51:38.857239 8487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:38.885149 8487 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:34631
I20260811 12:51:38.885190 8704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:34631 every 8 connection(s)
I20260811 12:51:38.886482 8487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:51:38.889583 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 8487
I20260811 12:51:38.889776 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:51:38.890354 7613 external_mini_cluster.cc:1428] Setting key 0d5e9c8269f11a9f2c073cd64556fbab
Aug 11 12:51:38 dist-test-slave-f0tn krb5kdc[7848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452698, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.67@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:51:38.906505 8710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:38.888939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:39263 (local address 127.7.111.126:41559)
0811 12:51:38.889395 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:38.889402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:38.892528 (+ 3126us) server_negotiation.cc:407] Connection header received
0811 12:51:38.892584 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:38.892588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:38.892652 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:38.892743 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:38.894477 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:38.895238 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:38.896751 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:38.897149 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:38.897270 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:38.900636 (+ 3366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:38.900640 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:38.900643 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:38.900677 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:38.903020 (+ 2343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:38.903749 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:38.903753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:38.903755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:38.903886 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:38.904380 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:38.904385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:38.904388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:38.904753 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:38.904908 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:38.905902 (+ 994us) server_negotiation.cc:299] Negotiation successful
0811 12:51:38.906126 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":205,"threads_started":1}
I20260811 12:51:38.907413 8711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:38.889493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:41559 (local address 127.7.111.67:39263)
0811 12:51:38.890321 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:38.890366 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:38.892254 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:38.893053 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:38.893064 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:38.893749 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:51:38.894311 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:38.894329 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:38.895388 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:38.895394 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:38.896608 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:38.896620 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:38.897367 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:38.897396 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:38.897637 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:38.898210 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:51:38.898232 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:51:38.900446 (+ 2214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:51:38.903153 (+ 2707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:51:38.903160 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:51:38.903172 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:51:38.903625 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:51:38.904002 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:51:38.904007 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:51:38.904011 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:51:38.904252 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:51:38.904901 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:51:38.904910 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:51:38.905441 (+ 531us) client_negotiation.cc:769] Sending connection context
0811 12:51:38.905781 (+ 340us) client_negotiation.cc:240] Negotiation successful
0811 12:51:38.906307 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":355,"threads_started":1}
I20260811 12:51:38.908645 8706 heartbeater.cc:347] Connected to a master server at 127.7.111.126:41559
I20260811 12:51:38.908938 8706 heartbeater.cc:464] Registering TS with master...
I20260811 12:51:38.909708 8706 heartbeater.cc:511] Master 127.7.111.126:41559 requested a full tablet report, sending...
I20260811 12:51:38.911078 7923 ts_manager.cc:194] Registered new tserver with Master: 4a980788662940379ffb7cc883ad2a5b (127.7.111.67:34631)
I20260811 12:51:38.911845 7923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.111.67@KRBTEST.COM'} at 127.7.111.67:39263
I20260811 12:51:38.917573 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:51:38.933911 8710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:38.921582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35828 (local address 127.7.111.126:41559)
0811 12:51:38.921744 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:38.921765 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:38.922140 (+ 375us) server_negotiation.cc:407] Connection header received
0811 12:51:38.922300 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:38.922306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:38.922365 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:38.922443 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:38.923825 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:38.924535 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:38.925856 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:38.926270 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:38.926376 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:38.928609 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:38.928614 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:38.928617 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:38.928646 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:38.930875 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:38.931717 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:38.931724 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:38.931728 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:38.931865 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:38.932330 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:38.932337 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:38.932339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:38.932698 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:38.932947 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:38.933339 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 12:51:38.933609 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 12:51:38.937780 7923 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35828:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:51:38.944536 7923 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:51:38.967649 8729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:38.955757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:34631 (local address 127.0.0.1:48888)
0811 12:51:38.956494 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:38.956550 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:38.956723 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:38.959235 (+ 2512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:38.959243 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:38.959276 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:38.959672 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:38.959683 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:38.961853 (+ 2170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:38.961859 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:38.964346 (+ 2487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:38.964363 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:38.966611 (+ 2248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:38.966618 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:38.966823 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:38.966875 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 12:51:38.967006 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 12:51:38.967196 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":344,"threads_started":1}
I20260811 12:51:38.968580 8731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:38.956036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48888 (local address 127.7.111.67:34631)
0811 12:51:38.958318 (+ 2282us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:38.958326 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:38.958359 (+ 33us) server_negotiation.cc:407] Connection header received
0811 12:51:38.958456 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:38.958464 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:38.958707 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:38.958906 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:38.959891 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:38.961629 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:38.964556 (+ 2927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:38.966075 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:38.966211 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:38.966455 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:38.968036 (+ 1581us) server_negotiation.cc:299] Negotiation successful
0811 12:51:38.968232 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2117,"thread_start_us":1954,"threads_started":1}
I20260811 12:51:38.972342 8598 tablet_service.cc:1467] Processing CreateTablet for tablet 9656e65c210f4626bf6a74201672fc03 (DEFAULT_TABLE table=test-table [id=5159834999444a1789714f8efb5fa123]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:51:38.973860 8598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9656e65c210f4626bf6a74201672fc03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:38.975801 8736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:38.955894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:36385 (local address 127.0.0.1:50062)
0811 12:51:38.967767 (+ 11873us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:38.967785 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:38.968472 (+ 687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:38.969576 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:38.969580 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:38.969606 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:38.969971 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:38.969982 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:38.971304 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:38.971309 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:38.973165 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:38.973179 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:38.975046 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:38.975053 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:38.975246 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:38.975278 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:51:38.975373 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 12:51:38.975495 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11743,"thread_start_us":13428,"threads_started":1}
I20260811 12:51:38.975912 8732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:38.956370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50062 (local address 127.7.111.65:36385)
0811 12:51:38.957820 (+ 1450us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:38.957830 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:38.967880 (+ 10050us) server_negotiation.cc:407] Connection header received
0811 12:51:38.968823 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:38.968832 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:38.969061 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:38.969233 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:38.970144 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:38.971169 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:38.973313 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:38.974802 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:38.974936 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:38.975162 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:38.975434 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 12:51:38.975604 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1265,"thread_start_us":1102,"threads_started":1}
I20260811 12:51:38.977634 8729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:38.957800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:37835 (local address 127.0.0.1:37980)
0811 12:51:38.968649 (+ 10849us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:38.968666 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:38.968749 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:38.969873 (+ 1124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:38.971304 (+ 1431us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:38.971335 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:38.971570 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:38.971582 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:38.973337 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:38.973343 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:38.975130 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:38.975147 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:38.976913 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:38.976919 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:38.977089 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:38.977122 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:51:38.977230 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 12:51:38.977364 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10754,"thread_start_us":10221,"threads_started":1}
I20260811 12:51:38.977666 8739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:38.958421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37980 (local address 127.7.111.66:37835)
0811 12:51:38.969405 (+ 10984us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:38.969413 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:38.969439 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:51:38.969533 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:38.969541 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:38.969694 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:38.969856 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:38.971734 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:38.973221 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:38.975304 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:38.976741 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:38.976886 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:38.977074 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:38.977254 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 12:51:38.977373 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10807,"thread_start_us":10691,"threads_started":1}
I20260811 12:51:38.979925 8145 tablet_service.cc:1467] Processing CreateTablet for tablet 9656e65c210f4626bf6a74201672fc03 (DEFAULT_TABLE table=test-table [id=5159834999444a1789714f8efb5fa123]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:51:38.981366 8145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9656e65c210f4626bf6a74201672fc03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:38.981308 8359 tablet_service.cc:1467] Processing CreateTablet for tablet 9656e65c210f4626bf6a74201672fc03 (DEFAULT_TABLE table=test-table [id=5159834999444a1789714f8efb5fa123]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:51:38.982663 8359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9656e65c210f4626bf6a74201672fc03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:38.992866 8743 tablet_bootstrap.cc:492] T 9656e65c210f4626bf6a74201672fc03 P 4a980788662940379ffb7cc883ad2a5b: Bootstrap starting.
I20260811 12:51:38.996665 8743 tablet_bootstrap.cc:654] T 9656e65c210f4626bf6a74201672fc03 P 4a980788662940379ffb7cc883ad2a5b: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:38.996703 8744 tablet_bootstrap.cc:492] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3: Bootstrap starting.
I20260811 12:51:38.998075 8746 tablet_bootstrap.cc:492] T 9656e65c210f4626bf6a74201672fc03 P e737e8637baa4b5aa70c3031df23d62a: Bootstrap starting.
I20260811 12:51:38.998124 8743 log.cc:824] T 9656e65c210f4626bf6a74201672fc03 P 4a980788662940379ffb7cc883ad2a5b: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:39.000676 8744 tablet_bootstrap.cc:654] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:39.001237 8746 tablet_bootstrap.cc:654] T 9656e65c210f4626bf6a74201672fc03 P e737e8637baa4b5aa70c3031df23d62a: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:39.001598 8743 tablet_bootstrap.cc:492] T 9656e65c210f4626bf6a74201672fc03 P 4a980788662940379ffb7cc883ad2a5b: No bootstrap required, opened a new log
I20260811 12:51:39.001685 8744 log.cc:824] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:39.001909 8743 ts_tablet_manager.cc:1397] T 9656e65c210f4626bf6a74201672fc03 P 4a980788662940379ffb7cc883ad2a5b: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260811 12:51:39.002493 8746 log.cc:824] T 9656e65c210f4626bf6a74201672fc03 P e737e8637baa4b5aa70c3031df23d62a: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:39.004278 8744 tablet_bootstrap.cc:492] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3: No bootstrap required, opened a new log
I20260811 12:51:39.004505 8744 ts_tablet_manager.cc:1397] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260811 12:51:39.005461 8746 tablet_bootstrap.cc:492] T 9656e65c210f4626bf6a74201672fc03 P e737e8637baa4b5aa70c3031df23d62a: No bootstrap required, opened a new log
I20260811 12:51:39.005770 8746 ts_tablet_manager.cc:1397] T 9656e65c210f4626bf6a74201672fc03 P e737e8637baa4b5aa70c3031df23d62a: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 12:51:39.012003 8744 raft_consensus.cc:348] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a980788662940379ffb7cc883ad2a5b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 34631 } } peers { permanent_uuid: "be1f4effe9b043869cccaac687eaf7b3" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36385 } } peers { permanent_uuid: "e737e8637baa4b5aa70c3031df23d62a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37835 } }
I20260811 12:51:39.012336 8744 raft_consensus.cc:374] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:39.012074 8743 raft_consensus.cc:348] T 9656e65c210f4626bf6a74201672fc03 P 4a980788662940379ffb7cc883ad2a5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a980788662940379ffb7cc883ad2a5b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 34631 } } peers { permanent_uuid: "be1f4effe9b043869cccaac687eaf7b3" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36385 } } peers { permanent_uuid: "e737e8637baa4b5aa70c3031df23d62a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37835 } }
I20260811 12:51:39.012461 8744 raft_consensus.cc:729] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be1f4effe9b043869cccaac687eaf7b3, State: Initialized, Role: FOLLOWER
I20260811 12:51:39.012475 8743 raft_consensus.cc:374] T 9656e65c210f4626bf6a74201672fc03 P 4a980788662940379ffb7cc883ad2a5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:39.012567 8743 raft_consensus.cc:729] T 9656e65c210f4626bf6a74201672fc03 P 4a980788662940379ffb7cc883ad2a5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a980788662940379ffb7cc883ad2a5b, State: Initialized, Role: FOLLOWER
I20260811 12:51:39.013026 8744 consensus_queue.cc:260] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a980788662940379ffb7cc883ad2a5b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 34631 } } peers { permanent_uuid: "be1f4effe9b043869cccaac687eaf7b3" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36385 } } peers { permanent_uuid: "e737e8637baa4b5aa70c3031df23d62a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37835 } }
I20260811 12:51:39.013303 8743 consensus_queue.cc:260] T 9656e65c210f4626bf6a74201672fc03 P 4a980788662940379ffb7cc883ad2a5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a980788662940379ffb7cc883ad2a5b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 34631 } } peers { permanent_uuid: "be1f4effe9b043869cccaac687eaf7b3" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36385 } } peers { permanent_uuid: "e737e8637baa4b5aa70c3031df23d62a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37835 } }
I20260811 12:51:39.014695 8744 ts_tablet_manager.cc:1428] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3: Time spent starting tablet: real 0.010s user 0.005s sys 0.006s
I20260811 12:51:39.016008 8746 raft_consensus.cc:348] T 9656e65c210f4626bf6a74201672fc03 P e737e8637baa4b5aa70c3031df23d62a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a980788662940379ffb7cc883ad2a5b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 34631 } } peers { permanent_uuid: "be1f4effe9b043869cccaac687eaf7b3" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36385 } } peers { permanent_uuid: "e737e8637baa4b5aa70c3031df23d62a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37835 } }
I20260811 12:51:39.016459 8746 raft_consensus.cc:374] T 9656e65c210f4626bf6a74201672fc03 P e737e8637baa4b5aa70c3031df23d62a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:39.016599 8746 raft_consensus.cc:729] T 9656e65c210f4626bf6a74201672fc03 P e737e8637baa4b5aa70c3031df23d62a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e737e8637baa4b5aa70c3031df23d62a, State: Initialized, Role: FOLLOWER
I20260811 12:51:39.017215 8746 consensus_queue.cc:260] T 9656e65c210f4626bf6a74201672fc03 P e737e8637baa4b5aa70c3031df23d62a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a980788662940379ffb7cc883ad2a5b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 34631 } } peers { permanent_uuid: "be1f4effe9b043869cccaac687eaf7b3" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36385 } } peers { permanent_uuid: "e737e8637baa4b5aa70c3031df23d62a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37835 } }
I20260811 12:51:39.018411 8478 heartbeater.cc:503] Master 127.7.111.126:41559 was elected leader, sending a full tablet report...
I20260811 12:51:39.019053 8746 ts_tablet_manager.cc:1428] T 9656e65c210f4626bf6a74201672fc03 P e737e8637baa4b5aa70c3031df23d62a: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260811 12:51:39.020438 8706 heartbeater.cc:503] Master 127.7.111.126:41559 was elected leader, sending a full tablet report...
I20260811 12:51:39.021059 8743 ts_tablet_manager.cc:1428] T 9656e65c210f4626bf6a74201672fc03 P 4a980788662940379ffb7cc883ad2a5b: Time spent starting tablet: real 0.019s user 0.013s sys 0.004s
W20260811 12:51:39.056392 8247 tablet.cc:2369] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:51:39.107266 8481 tablet.cc:2369] T 9656e65c210f4626bf6a74201672fc03 P e737e8637baa4b5aa70c3031df23d62a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:51:39.138119 8708 tablet.cc:2369] T 9656e65c210f4626bf6a74201672fc03 P 4a980788662940379ffb7cc883ad2a5b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:51:39.159776 8752 raft_consensus.cc:482] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:51:39.160171 8752 raft_consensus.cc:504] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a980788662940379ffb7cc883ad2a5b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 34631 } } peers { permanent_uuid: "be1f4effe9b043869cccaac687eaf7b3" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36385 } } peers { permanent_uuid: "e737e8637baa4b5aa70c3031df23d62a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37835 } }
I20260811 12:51:39.162410 8752 leader_election.cc:288] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a980788662940379ffb7cc883ad2a5b (127.7.111.67:34631), e737e8637baa4b5aa70c3031df23d62a (127.7.111.66:37835)
I20260811 12:51:39.170347 8731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:39.163423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:33983 (local address 127.7.111.67:34631)
0811 12:51:39.163631 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:39.163636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:39.164420 (+ 784us) server_negotiation.cc:407] Connection header received
0811 12:51:39.164588 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:39.164592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:39.164651 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:39.164735 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:39.165367 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:39.166503 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:39.168631 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:39.169434 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:39.169590 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:39.169665 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:39.169947 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 12:51:39.170079 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:51:39.170377 8755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:39.162752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:34631 (local address 127.7.111.65:33983)
0811 12:51:39.164305 (+ 1553us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:39.164320 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:39.164449 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:39.164774 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:39.164779 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:39.164802 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:39.165185 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:39.165196 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:39.166648 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:39.166654 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:39.168479 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:39.168490 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:39.169625 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:39.169631 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:39.169795 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:39.169824 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 12:51:39.169909 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 12:51:39.170041 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1462,"thread_start_us":893,"threads_started":1}
I20260811 12:51:39.170598 8739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:39.162985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:53199 (local address 127.7.111.66:37835)
0811 12:51:39.163187 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:39.163193 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:39.163781 (+ 588us) server_negotiation.cc:407] Connection header received
0811 12:51:39.164093 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:39.164099 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:39.164173 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:39.164230 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:39.165015 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:39.165886 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:39.168165 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:39.169431 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:39.169576 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:39.169650 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:39.170201 (+ 551us) server_negotiation.cc:299] Negotiation successful
0811 12:51:39.170341 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:51:39.171141 8757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:39.162646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:37835 (local address 127.7.111.65:53199)
0811 12:51:39.163651 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:39.163671 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:39.163914 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:39.164421 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:39.164426 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:39.164452 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:39.164840 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:39.164852 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:39.166015 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:39.166020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:39.167991 (+ 1971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:39.168006 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:39.169822 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:39.169828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:39.170016 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:39.170049 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:51:39.170725 (+ 676us) client_negotiation.cc:240] Negotiation successful
0811 12:51:39.170857 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":872,"mutex_wait_us":1,"thread_start_us":1389,"threads_started":1}
I20260811 12:51:39.171779 8625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9656e65c210f4626bf6a74201672fc03" candidate_uuid: "be1f4effe9b043869cccaac687eaf7b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a980788662940379ffb7cc883ad2a5b" is_pre_election: true
I20260811 12:51:39.172355 8625 raft_consensus.cc:2454] T 9656e65c210f4626bf6a74201672fc03 P 4a980788662940379ffb7cc883ad2a5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be1f4effe9b043869cccaac687eaf7b3 in term 0.
I20260811 12:51:39.173127 8059 leader_election.cc:302] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a980788662940379ffb7cc883ad2a5b, be1f4effe9b043869cccaac687eaf7b3; no voters:
I20260811 12:51:39.173202 8389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9656e65c210f4626bf6a74201672fc03" candidate_uuid: "be1f4effe9b043869cccaac687eaf7b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e737e8637baa4b5aa70c3031df23d62a" is_pre_election: true
I20260811 12:51:39.173623 8752 raft_consensus.cc:2781] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:51:39.173756 8752 raft_consensus.cc:482] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:51:39.173767 8389 raft_consensus.cc:2454] T 9656e65c210f4626bf6a74201672fc03 P e737e8637baa4b5aa70c3031df23d62a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be1f4effe9b043869cccaac687eaf7b3 in term 0.
I20260811 12:51:39.173849 8752 raft_consensus.cc:3037] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:39.176029 8752 raft_consensus.cc:504] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a980788662940379ffb7cc883ad2a5b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 34631 } } peers { permanent_uuid: "be1f4effe9b043869cccaac687eaf7b3" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36385 } } peers { permanent_uuid: "e737e8637baa4b5aa70c3031df23d62a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37835 } }
I20260811 12:51:39.176600 8752 leader_election.cc:288] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3 [CANDIDATE]: Term 1 election: Requested vote from peers 4a980788662940379ffb7cc883ad2a5b (127.7.111.67:34631), e737e8637baa4b5aa70c3031df23d62a (127.7.111.66:37835)
I20260811 12:51:39.177109 8625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9656e65c210f4626bf6a74201672fc03" candidate_uuid: "be1f4effe9b043869cccaac687eaf7b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a980788662940379ffb7cc883ad2a5b"
I20260811 12:51:39.177110 8389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9656e65c210f4626bf6a74201672fc03" candidate_uuid: "be1f4effe9b043869cccaac687eaf7b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e737e8637baa4b5aa70c3031df23d62a"
I20260811 12:51:39.177325 8389 raft_consensus.cc:3037] T 9656e65c210f4626bf6a74201672fc03 P e737e8637baa4b5aa70c3031df23d62a [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:39.177325 8625 raft_consensus.cc:3037] T 9656e65c210f4626bf6a74201672fc03 P 4a980788662940379ffb7cc883ad2a5b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:39.180487 8389 raft_consensus.cc:2454] T 9656e65c210f4626bf6a74201672fc03 P e737e8637baa4b5aa70c3031df23d62a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be1f4effe9b043869cccaac687eaf7b3 in term 1.
I20260811 12:51:39.180490 8625 raft_consensus.cc:2454] T 9656e65c210f4626bf6a74201672fc03 P 4a980788662940379ffb7cc883ad2a5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be1f4effe9b043869cccaac687eaf7b3 in term 1.
I20260811 12:51:39.180928 8059 leader_election.cc:302] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a980788662940379ffb7cc883ad2a5b, be1f4effe9b043869cccaac687eaf7b3; no voters:
I20260811 12:51:39.181236 8752 raft_consensus.cc:2781] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:51:39.181696 8752 raft_consensus.cc:686] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3 [term 1 LEADER]: Becoming Leader. State: Replica: be1f4effe9b043869cccaac687eaf7b3, State: Running, Role: LEADER
I20260811 12:51:39.182271 8752 consensus_queue.cc:237] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a980788662940379ffb7cc883ad2a5b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 34631 } } peers { permanent_uuid: "be1f4effe9b043869cccaac687eaf7b3" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36385 } } peers { permanent_uuid: "e737e8637baa4b5aa70c3031df23d62a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37835 } }
I20260811 12:51:39.186975 7923 catalog_manager.cc:5495] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to be1f4effe9b043869cccaac687eaf7b3 (127.7.111.65). New cstate: current_term: 1 leader_uuid: "be1f4effe9b043869cccaac687eaf7b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a980788662940379ffb7cc883ad2a5b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 34631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be1f4effe9b043869cccaac687eaf7b3" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 36385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e737e8637baa4b5aa70c3031df23d62a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37835 } health_report { overall_health: UNKNOWN } } }
I20260811 12:51:39.203222 8732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:39.198047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50076 (local address 127.7.111.65:36385)
0811 12:51:39.198234 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:39.198239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:39.198350 (+ 111us) server_negotiation.cc:407] Connection header received
0811 12:51:39.198542 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:39.198545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:39.198625 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:39.198750 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:51:39.199037 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:39.199849 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:39.201263 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:39.201727 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:39.201843 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:39.202264 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:51:39.202781 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:51:39.202895 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:39.202932 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 12:51:39.202996 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:51:39.211632 8389 raft_consensus.cc:1261] T 9656e65c210f4626bf6a74201672fc03 P e737e8637baa4b5aa70c3031df23d62a [term 1 FOLLOWER]: Refusing update from remote peer be1f4effe9b043869cccaac687eaf7b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:51:39.211632 8625 raft_consensus.cc:1261] T 9656e65c210f4626bf6a74201672fc03 P 4a980788662940379ffb7cc883ad2a5b [term 1 FOLLOWER]: Refusing update from remote peer be1f4effe9b043869cccaac687eaf7b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:51:39.212778 8758 consensus_queue.cc:1035] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a980788662940379ffb7cc883ad2a5b" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 34631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:51:39.213235 8752 consensus_queue.cc:1035] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e737e8637baa4b5aa70c3031df23d62a" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 37835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:51:39.242583 7921 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:51:39.242901 7921 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:51:39.246356 7921 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6788004703114cbcbffb18842273ab65: Sending DeleteTablet for 3 replicas of tablet 9656e65c210f4626bf6a74201672fc03
I20260811 12:51:39.247458 8145 tablet_service.cc:1514] Processing DeleteTablet for tablet 9656e65c210f4626bf6a74201672fc03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:51:39 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:50062
I20260811 12:51:39.247468 8598 tablet_service.cc:1514] Processing DeleteTablet for tablet 9656e65c210f4626bf6a74201672fc03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:51:39 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:48888
I20260811 12:51:39.247515 8359 tablet_service.cc:1514] Processing DeleteTablet for tablet 9656e65c210f4626bf6a74201672fc03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:51:39 UTC) from {username='kudu', principal='kudu/127.7.111.126@KRBTEST.COM'} at 127.0.0.1:37980
I20260811 12:51:39.249547 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 8030
I20260811 12:51:39.276736 8779 tablet_replica.cc:330] T 9656e65c210f4626bf6a74201672fc03 P 4a980788662940379ffb7cc883ad2a5b: stopping tablet replica
I20260811 12:51:39.280695 8779 raft_consensus.cc:2229] T 9656e65c210f4626bf6a74201672fc03 P 4a980788662940379ffb7cc883ad2a5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:51:39.281546 8779 raft_consensus.cc:2258] T 9656e65c210f4626bf6a74201672fc03 P 4a980788662940379ffb7cc883ad2a5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:51:39.282241 8778 tablet_replica.cc:330] T 9656e65c210f4626bf6a74201672fc03 P e737e8637baa4b5aa70c3031df23d62a: stopping tablet replica
I20260811 12:51:39.282708 8778 raft_consensus.cc:2229] T 9656e65c210f4626bf6a74201672fc03 P e737e8637baa4b5aa70c3031df23d62a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:51:39.283190 8778 raft_consensus.cc:2258] T 9656e65c210f4626bf6a74201672fc03 P e737e8637baa4b5aa70c3031df23d62a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:51:39.284008 8777 tablet_replica.cc:330] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3: stopping tablet replica
I20260811 12:51:39.284428 8777 raft_consensus.cc:2229] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:51:39.285471 8778 ts_tablet_manager.cc:1905] T 9656e65c210f4626bf6a74201672fc03 P e737e8637baa4b5aa70c3031df23d62a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:51:39.288607 8779 ts_tablet_manager.cc:1905] T 9656e65c210f4626bf6a74201672fc03 P 4a980788662940379ffb7cc883ad2a5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:51:39.290577 8778 ts_tablet_manager.cc:1918] T 9656e65c210f4626bf6a74201672fc03 P e737e8637baa4b5aa70c3031df23d62a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:51:39.290735 8778 log.cc:1196] T 9656e65c210f4626bf6a74201672fc03 P e737e8637baa4b5aa70c3031df23d62a: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/9656e65c210f4626bf6a74201672fc03
I20260811 12:51:39.291168 8778 ts_tablet_manager.cc:1939] T 9656e65c210f4626bf6a74201672fc03 P e737e8637baa4b5aa70c3031df23d62a: Deleting consensus metadata
I20260811 12:51:39.292730 7891 catalog_manager.cc:4841] TS e737e8637baa4b5aa70c3031df23d62a (127.7.111.66:37835): tablet 9656e65c210f4626bf6a74201672fc03 (table test-table [id=5159834999444a1789714f8efb5fa123]) successfully deleted
I20260811 12:51:39.293514 8777 raft_consensus.cc:2258] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:51:39.295578 8779 ts_tablet_manager.cc:1918] T 9656e65c210f4626bf6a74201672fc03 P 4a980788662940379ffb7cc883ad2a5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:51:39.295770 8779 log.cc:1196] T 9656e65c210f4626bf6a74201672fc03 P 4a980788662940379ffb7cc883ad2a5b: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/9656e65c210f4626bf6a74201672fc03
I20260811 12:51:39.296088 8777 ts_tablet_manager.cc:1905] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:51:39.296308 8779 ts_tablet_manager.cc:1939] T 9656e65c210f4626bf6a74201672fc03 P 4a980788662940379ffb7cc883ad2a5b: Deleting consensus metadata
Aug 11 12:51:39 dist-test-slave-f0tn krb5kdc[7848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.65@KRBTEST.COM
I20260811 12:51:39.298184 7892 catalog_manager.cc:4841] TS 4a980788662940379ffb7cc883ad2a5b (127.7.111.67:34631): tablet 9656e65c210f4626bf6a74201672fc03 (table test-table [id=5159834999444a1789714f8efb5fa123]) successfully deleted
I20260811 12:51:39.303272 8777 ts_tablet_manager.cc:1918] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:51:39.303455 8777 log.cc:1196] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/9656e65c210f4626bf6a74201672fc03
I20260811 12:51:39.304001 8777 ts_tablet_manager.cc:1939] T 9656e65c210f4626bf6a74201672fc03 P be1f4effe9b043869cccaac687eaf7b3: Deleting consensus metadata
I20260811 12:51:39.305751 7893 catalog_manager.cc:4841] TS be1f4effe9b043869cccaac687eaf7b3 (127.7.111.65:36385): tablet 9656e65c210f4626bf6a74201672fc03 (table test-table [id=5159834999444a1789714f8efb5fa123]) successfully deleted
I20260811 12:51:39.305863 8732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:39.250833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50084 (local address 127.7.111.65:36385)
0811 12:51:39.251025 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:39.251031 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:39.273562 (+ 22531us) server_negotiation.cc:407] Connection header received
0811 12:51:39.274154 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:39.274161 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:39.274260 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:39.274341 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:39.276997 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:39.278292 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:39.284982 (+ 6690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:39.285775 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:39.285879 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:39.297872 (+ 11993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:39.297890 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:39.297909 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:39.297984 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:39.302336 (+ 4352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:39.303259 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:39.303278 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:39.303281 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:39.303480 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:39.304015 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:39.304020 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:39.304024 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:39.304505 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:39.304869 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:39.305215 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 12:51:39.305493 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:51:39.306859 8235 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:39.407776 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 8030
I20260811 12:51:39.431730 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 8254
Aug 11 12:51:39 dist-test-slave-f0tn krb5kdc[7848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.66@KRBTEST.COM
I20260811 12:51:39.446010 8739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:39.432563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37996 (local address 127.7.111.66:37835)
0811 12:51:39.432743 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:39.432747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:39.432766 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:51:39.432889 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:39.432893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:39.432962 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:39.433043 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:39.434347 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:39.435099 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:39.436310 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:39.436687 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:39.436775 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:39.440861 (+ 4086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:39.440876 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:39.440893 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:39.440931 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:39.443594 (+ 2663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:39.444225 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:39.444233 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:39.444235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:39.444355 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:39.444637 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:39.444640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:39.444642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:39.445055 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:39.445270 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:39.445531 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 12:51:39.445705 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:51:39.446686 8470 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:39.534898 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 8254
I20260811 12:51:39.563469 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 8487
Aug 11 12:51:39 dist-test-slave-f0tn krb5kdc[7848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.67@KRBTEST.COM
I20260811 12:51:39.587978 8731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:39.564240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48900 (local address 127.7.111.67:34631)
0811 12:51:39.564412 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:39.564417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:39.564435 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:51:39.564567 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:39.564571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:39.564635 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:39.564721 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:39.566046 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:39.566756 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:39.567912 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:39.568294 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:39.568382 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:39.582890 (+ 14508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:39.582909 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:39.582929 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:39.582967 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:39.585531 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:39.586159 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:39.586166 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:39.586168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:39.586294 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:39.586654 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:39.586659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:39.586661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:39.587036 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:39.587282 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:39.587487 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 12:51:39.587655 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:51:39.588611 8695 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:39.677263 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 8487
I20260811 12:51:39.703318 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 7870
I20260811 12:51:39.704437 7975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:39.813586 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 7870
2026-08-11T12:51:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4733 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T12:51:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:51:39Z Disabled control of system clock
I20260811 12:51:39.916880 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:44719
--webserver_interface=127.7.111.126
--webserver_port=0
--builtin_ntp_servers=127.7.111.84:44481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:44719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:51:40.251322 8795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:40.251627 8795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:40.251716 8795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:40.258869 8795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:51:40.258962 8795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:40.259006 8795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:40.259060 8795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:51:40.259104 8795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:51:40.268100 8795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:44481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:44719
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:44719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:40.269356 8795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:40.271138 8795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:40.282799 8809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:40.282881 8812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:40.282847 8810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:40.283907 8795 server_base.cc:1034] running on GCE node
I20260811 12:51:40.284687 8795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:40.286098 8795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:40.287298 8795 hybrid_clock.cc:648] HybridClock initialized: now 1786452700287256 us; error 37 us; skew 500 ppm
I20260811 12:51:40.287632 8795 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:51:40.289863 8795 webserver.cc:460] Webserver started at http://127.7.111.126:40333/ using document root <none> and password file <none>
I20260811 12:51:40.290541 8795 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:40.290647 8795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:40.290962 8795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:40.293854 8795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "8e77597fad124e82b082e61d72779b8f"
format_stamp: "Formatted at 2026-08-11 12:51:40 on dist-test-slave-f0tn"
server_key: "b770a341564a9bce0a53087d7edf4a4b"
server_key_iv: "7c4d65d4c552d1373e5e45b96d2c3498"
server_key_version: "encryptionkey@0"
I20260811 12:51:40.294613 8795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "8e77597fad124e82b082e61d72779b8f"
format_stamp: "Formatted at 2026-08-11 12:51:40 on dist-test-slave-f0tn"
server_key: "b770a341564a9bce0a53087d7edf4a4b"
server_key_iv: "7c4d65d4c552d1373e5e45b96d2c3498"
server_key_version: "encryptionkey@0"
I20260811 12:51:40.299505 8795 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 12:51:40.302812 8821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:40.304064 8795 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:51:40.304244 8795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "8e77597fad124e82b082e61d72779b8f"
format_stamp: "Formatted at 2026-08-11 12:51:40 on dist-test-slave-f0tn"
server_key: "b770a341564a9bce0a53087d7edf4a4b"
server_key_iv: "7c4d65d4c552d1373e5e45b96d2c3498"
server_key_version: "encryptionkey@0"
I20260811 12:51:40.304440 8795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:40.328878 8795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:40.329820 8795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:40.330101 8795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:40.348677 8795 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:44719
I20260811 12:51:40.348718 8923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:44719 every 8 connection(s)
I20260811 12:51:40.350209 8795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:51:40.354139 8924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:40.355422 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 8795
I20260811 12:51:40.355604 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:51:40.356073 7613 external_mini_cluster.cc:1428] Setting key 9d5a896b7c60b1e42079225754f56061
I20260811 12:51:40.366101 8924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f: Bootstrap starting.
I20260811 12:51:40.369705 8924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:40.370507 8927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:40.358097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48960 (local address 127.7.111.126:44719)
0811 12:51:40.359241 (+ 1144us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:40.359258 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:40.359312 (+ 54us) server_negotiation.cc:407] Connection header received
0811 12:51:40.361174 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:40.361201 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:40.361721 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:40.362159 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:40.362784 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:40.364137 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:40.366084 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:40.366829 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:40.366965 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:40.368167 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:40.368182 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:40.368203 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:40.368720 (+ 517us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:40.368815 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:40.368840 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:40.368925 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:40.369149 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 12:51:40.369507 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":421,"threads_started":1}
I20260811 12:51:40.370820 8924 log.cc:824] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:40.373812 8924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f: No bootstrap required, opened a new log
I20260811 12:51:40.381091 8924 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e77597fad124e82b082e61d72779b8f" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 44719 } }
I20260811 12:51:40.381479 8924 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:40.381590 8924 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e77597fad124e82b082e61d72779b8f, State: Initialized, Role: FOLLOWER
I20260811 12:51:40.382133 8924 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e77597fad124e82b082e61d72779b8f" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 44719 } }
I20260811 12:51:40.382360 8924 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:51:40.382488 8924 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:51:40.382633 8924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:40.384979 8924 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e77597fad124e82b082e61d72779b8f" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 44719 } }
I20260811 12:51:40.385488 8924 leader_election.cc:302] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e77597fad124e82b082e61d72779b8f; no voters:
I20260811 12:51:40.386040 8924 leader_election.cc:288] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:51:40.386185 8935 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:51:40.387285 8935 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f [term 1 LEADER]: Becoming Leader. State: Replica: 8e77597fad124e82b082e61d72779b8f, State: Running, Role: LEADER
I20260811 12:51:40.387782 8935 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e77597fad124e82b082e61d72779b8f" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 44719 } }
I20260811 12:51:40.388095 8924 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:51:40.390627 8941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e77597fad124e82b082e61d72779b8f. Latest consensus state: current_term: 1 leader_uuid: "8e77597fad124e82b082e61d72779b8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e77597fad124e82b082e61d72779b8f" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 44719 } } }
I20260811 12:51:40.390713 8938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e77597fad124e82b082e61d72779b8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e77597fad124e82b082e61d72779b8f" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 44719 } } }
I20260811 12:51:40.391129 8938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:40.391137 8941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:40.392088 8953 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:51:40.397763 8953 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:51:40.407908 8953 catalog_manager.cc:1347] Generated new cluster ID: 0ee4b5eaa7f24d17ab26d188b540b6b5
I20260811 12:51:40.408031 8953 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:51:40.458456 8953 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:51:40.459759 8953 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:51:40.468560 8953 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f: Generated new TSK 0
I20260811 12:51:40.469345 8953 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:51:40.474007 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.65:0
--local_ip_for_outbound_sockets=127.7.111.65
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:44719
--builtin_ntp_servers=127.7.111.84:44481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:51:40.840363 8969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:40.840631 8969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:40.840718 8969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:40.847919 8969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:40.848026 8969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:40.848151 8969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.65
I20260811 12:51:40.857417 8969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:44481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.111.65
--webserver_port=0
--tserver_master_addrs=127.7.111.126:44719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.65
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:40.858680 8969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:40.860482 8969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:40.873801 8979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:40.874063 8977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:40.873807 8982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:40.874732 8969 server_base.cc:1034] running on GCE node
I20260811 12:51:40.875291 8969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:40.876183 8969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:40.877406 8969 hybrid_clock.cc:648] HybridClock initialized: now 1786452700877333 us; error 38 us; skew 500 ppm
I20260811 12:51:40.877725 8969 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:51:40.879957 8969 webserver.cc:460] Webserver started at http://127.7.111.65:42203/ using document root <none> and password file <none>
I20260811 12:51:40.880599 8969 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:40.880702 8969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:40.881012 8969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:40.883776 8969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-0/data/instance:
uuid: "9c15ff0dff734f3589035d74cf01060b"
format_stamp: "Formatted at 2026-08-11 12:51:40 on dist-test-slave-f0tn"
server_key: "ce21adf57cf3cd4612da946c02c81476"
server_key_iv: "a93bb813b78d200e0d8d6ed8c7030e45"
server_key_version: "encryptionkey@0"
I20260811 12:51:40.884570 8969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance:
uuid: "9c15ff0dff734f3589035d74cf01060b"
format_stamp: "Formatted at 2026-08-11 12:51:40 on dist-test-slave-f0tn"
server_key: "ce21adf57cf3cd4612da946c02c81476"
server_key_iv: "a93bb813b78d200e0d8d6ed8c7030e45"
server_key_version: "encryptionkey@0"
I20260811 12:51:40.889439 8969 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.004s
I20260811 12:51:40.892876 8993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:40.894192 8969 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:51:40.894338 8969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-0/wal
uuid: "9c15ff0dff734f3589035d74cf01060b"
format_stamp: "Formatted at 2026-08-11 12:51:40 on dist-test-slave-f0tn"
server_key: "ce21adf57cf3cd4612da946c02c81476"
server_key_iv: "a93bb813b78d200e0d8d6ed8c7030e45"
server_key_version: "encryptionkey@0"
I20260811 12:51:40.894527 8969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:40.920715 8969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:40.921556 8969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:40.921860 8969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:40.922825 8969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:51:40.924283 8969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:51:40.924371 8969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:40.924475 8969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:51:40.924552 8969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:40.954594 8969 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.65:39371
I20260811 12:51:40.954687 9187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.65:39371 every 8 connection(s)
I20260811 12:51:40.955883 8969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-0/data/info.pb
I20260811 12:51:40.964399 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 8969
I20260811 12:51:40.964584 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-0/wal/instance
I20260811 12:51:40.965155 7613 external_mini_cluster.cc:1428] Setting key e40b87df56d9e76c38f0be4628e23e5c
I20260811 12:51:40.968279 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.66:0
--local_ip_for_outbound_sockets=127.7.111.66
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:44719
--builtin_ntp_servers=127.7.111.84:44481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:51:40.968523 9190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:40.958355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:49309 (local address 127.7.111.126:44719)
0811 12:51:40.958851 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:40.958858 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:40.959991 (+ 1133us) server_negotiation.cc:407] Connection header received
0811 12:51:40.961730 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:40.961734 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:40.961826 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:40.961923 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:40.962898 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:40.963620 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:40.965729 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:40.966248 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:40.966376 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:40.967720 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:40.967724 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:40.967728 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:40.967745 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:40.967788 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:40.967799 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:40.967871 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:40.968142 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 12:51:40.968229 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":229,"threads_started":1}
I20260811 12:51:40.969882 9191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:40.958935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:44719 (local address 127.7.111.65:49309)
0811 12:51:40.959837 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:40.959873 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:40.961500 (+ 1627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:40.962097 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:40.962106 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:40.962194 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:51:40.962765 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:40.962787 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:40.963729 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:40.963733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:40.965586 (+ 1853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:40.965601 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:40.966448 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:40.966454 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:40.966646 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:40.967373 (+ 727us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:51:40.967376 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:51:40.967428 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:51:40.967535 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:51:40.967895 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:51:40.967908 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:51:40.967917 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 12:51:40.968103 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 12:51:40.968504 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":280,"threads_started":1}
I20260811 12:51:40.971375 9188 heartbeater.cc:347] Connected to a master server at 127.7.111.126:44719
I20260811 12:51:40.971668 9188 heartbeater.cc:464] Registering TS with master...
I20260811 12:51:40.972496 9188 heartbeater.cc:511] Master 127.7.111.126:44719 requested a full tablet report, sending...
I20260811 12:51:40.974484 8847 ts_manager.cc:194] Registered new tserver with Master: 9c15ff0dff734f3589035d74cf01060b (127.7.111.65:39371)
I20260811 12:51:40.975898 8847 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.111.65:49309
W20260811 12:51:41.309513 9192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:41.309814 9192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:41.309913 9192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:41.317165 9192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:41.317281 9192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:41.317418 9192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.66
I20260811 12:51:41.326740 9192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:44481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.111.66
--webserver_port=0
--tserver_master_addrs=127.7.111.126:44719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.66
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:41.328090 9192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:41.329939 9192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:41.342542 9200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:41.343284 9201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:41.343308 9204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:41.343668 9192 server_base.cc:1034] running on GCE node
I20260811 12:51:41.344328 9192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:41.345247 9192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:41.346452 9192 hybrid_clock.cc:648] HybridClock initialized: now 1786452701346412 us; error 56 us; skew 500 ppm
I20260811 12:51:41.346760 9192 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:51:41.349120 9192 webserver.cc:460] Webserver started at http://127.7.111.66:36797/ using document root <none> and password file <none>
I20260811 12:51:41.349833 9192 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:41.349938 9192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:41.350251 9192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:41.353037 9192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-1/data/instance:
uuid: "9d21a743d35345618ba961f8dae9400e"
format_stamp: "Formatted at 2026-08-11 12:51:41 on dist-test-slave-f0tn"
server_key: "72be83955c6885ac3034761288c202b0"
server_key_iv: "0dbd5e009db1c1652a57d2dc60fea0ea"
server_key_version: "encryptionkey@0"
I20260811 12:51:41.353847 9192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance:
uuid: "9d21a743d35345618ba961f8dae9400e"
format_stamp: "Formatted at 2026-08-11 12:51:41 on dist-test-slave-f0tn"
server_key: "72be83955c6885ac3034761288c202b0"
server_key_iv: "0dbd5e009db1c1652a57d2dc60fea0ea"
server_key_version: "encryptionkey@0"
I20260811 12:51:41.358750 9192 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 12:51:41.362144 9217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:41.363389 9192 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:51:41.363544 9192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-1/wal
uuid: "9d21a743d35345618ba961f8dae9400e"
format_stamp: "Formatted at 2026-08-11 12:51:41 on dist-test-slave-f0tn"
server_key: "72be83955c6885ac3034761288c202b0"
server_key_iv: "0dbd5e009db1c1652a57d2dc60fea0ea"
server_key_version: "encryptionkey@0"
I20260811 12:51:41.363745 9192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:41.381152 9192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:41.382157 9192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:41.382454 9192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:41.383427 9192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:51:41.384876 9192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:51:41.384979 9192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:41.385083 9192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:51:41.385160 9192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:41.415560 9192 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.66:39703
I20260811 12:51:41.415611 9405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.66:39703 every 8 connection(s)
I20260811 12:51:41.416848 9192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-1/data/info.pb
I20260811 12:51:41.422334 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 9192
I20260811 12:51:41.422492 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-1/wal/instance
I20260811 12:51:41.423095 7613 external_mini_cluster.cc:1428] Setting key 5894a9bf7642af861a1e5c38a2e8289a
I20260811 12:51:41.425913 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.111.67:0
--local_ip_for_outbound_sockets=127.7.111.67
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:44719
--builtin_ntp_servers=127.7.111.84:44481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:51:41.429005 9190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:41.419300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:54787 (local address 127.7.111.126:44719)
0811 12:51:41.419527 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:41.419533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:41.420919 (+ 1386us) server_negotiation.cc:407] Connection header received
0811 12:51:41.422706 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:41.422712 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:41.422800 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:41.422917 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:41.424035 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:41.424808 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:41.426428 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:41.426950 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:41.427078 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:41.428285 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:41.428290 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:41.428294 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:41.428314 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:41.428354 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:41.428368 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:41.428446 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:41.428617 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 12:51:41.428715 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 12:51:41.429915 9408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:41.419876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:44719 (local address 127.7.111.66:54787)
0811 12:51:41.420753 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:41.420795 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:41.422422 (+ 1627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:41.423101 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:41.423108 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:41.423191 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:51:41.423883 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:41.423905 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:41.424957 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:41.424962 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:41.426288 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:41.426300 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:41.427139 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:41.427144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:41.427342 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:41.427986 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:51:41.427988 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:51:41.428026 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:51:41.428100 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:51:41.428452 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:51:41.428458 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:51:41.428463 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 12:51:41.428589 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 12:51:41.428864 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":271,"threads_started":1}
I20260811 12:51:41.431104 9406 heartbeater.cc:347] Connected to a master server at 127.7.111.126:44719
I20260811 12:51:41.431421 9406 heartbeater.cc:464] Registering TS with master...
I20260811 12:51:41.432296 9406 heartbeater.cc:511] Master 127.7.111.126:44719 requested a full tablet report, sending...
I20260811 12:51:41.433818 8847 ts_manager.cc:194] Registered new tserver with Master: 9d21a743d35345618ba961f8dae9400e (127.7.111.66:39703)
I20260811 12:51:41.434592 8847 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.111.66:54787
W20260811 12:51:41.767956 9409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:41.768270 9409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:41.768400 9409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:41.775611 9409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:41.775709 9409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:41.775812 9409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.111.67
I20260811 12:51:41.784533 9409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:44481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.111.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.111.67
--webserver_port=0
--tserver_master_addrs=127.7.111.126:44719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.111.67
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:41.785766 9409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:41.787532 9409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:41.800619 9424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:41.800715 9414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:41.800737 9416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:41.801357 9409 server_base.cc:1034] running on GCE node
I20260811 12:51:41.802021 9409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:41.802932 9409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:41.804128 9409 hybrid_clock.cc:648] HybridClock initialized: now 1786452701804084 us; error 36 us; skew 500 ppm
I20260811 12:51:41.804440 9409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:51:41.806589 9409 webserver.cc:460] Webserver started at http://127.7.111.67:36859/ using document root <none> and password file <none>
I20260811 12:51:41.807263 9409 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:41.807353 9409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:41.807673 9409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:41.810622 9409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-2/data/instance:
uuid: "5ab4dc0ae95a4c0886b24a4b4f6fdeac"
format_stamp: "Formatted at 2026-08-11 12:51:41 on dist-test-slave-f0tn"
server_key: "93b2f9969c89effc818c6c85411ba91f"
server_key_iv: "edb7ecfb13537c53ae0b533643d33b38"
server_key_version: "encryptionkey@0"
I20260811 12:51:41.811537 9409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance:
uuid: "5ab4dc0ae95a4c0886b24a4b4f6fdeac"
format_stamp: "Formatted at 2026-08-11 12:51:41 on dist-test-slave-f0tn"
server_key: "93b2f9969c89effc818c6c85411ba91f"
server_key_iv: "edb7ecfb13537c53ae0b533643d33b38"
server_key_version: "encryptionkey@0"
I20260811 12:51:41.816632 9409 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:51:41.820055 9431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:41.821336 9409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:51:41.821522 9409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-2/wal
uuid: "5ab4dc0ae95a4c0886b24a4b4f6fdeac"
format_stamp: "Formatted at 2026-08-11 12:51:41 on dist-test-slave-f0tn"
server_key: "93b2f9969c89effc818c6c85411ba91f"
server_key_iv: "edb7ecfb13537c53ae0b533643d33b38"
server_key_version: "encryptionkey@0"
I20260811 12:51:41.821728 9409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:41.833060 9409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:41.833952 9409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:41.834242 9409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:41.835251 9409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:51:41.836632 9409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:51:41.836724 9409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:41.836833 9409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:51:41.836905 9409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:41.867118 9409 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.67:33479
I20260811 12:51:41.867169 9632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.67:33479 every 8 connection(s)
I20260811 12:51:41.868496 9409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-2/data/info.pb
I20260811 12:51:41.869365 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 9409
I20260811 12:51:41.869553 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-2/wal/instance
I20260811 12:51:41.870138 7613 external_mini_cluster.cc:1428] Setting key b998d3bcb6a3c5d6aba646af6b318335
I20260811 12:51:41.881301 9190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:41.871467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.67:39783 (local address 127.7.111.126:44719)
0811 12:51:41.871655 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:41.871661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:41.873352 (+ 1691us) server_negotiation.cc:407] Connection header received
0811 12:51:41.875207 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:41.875212 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:41.875295 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:41.875381 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:41.876422 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:41.877145 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:41.878688 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:41.879112 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:41.879234 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:41.880507 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:41.880512 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:41.880515 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:41.880532 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:41.880564 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:41.880575 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:41.880647 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:41.880865 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 12:51:41.880961 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:51:41.882246 9637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:41.872193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:44719 (local address 127.7.111.67:39783)
0811 12:51:41.873222 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:41.873261 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:41.874947 (+ 1686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:41.875616 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:41.875624 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:41.875722 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:51:41.876261 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:41.876277 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:41.877307 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:41.877311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:41.878545 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:41.878558 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:41.879331 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:41.879339 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:41.879558 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:41.880195 (+ 637us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:51:41.880198 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:51:41.880239 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:51:41.880310 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:51:41.880663 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:51:41.880682 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:51:41.880688 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 12:51:41.880844 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 12:51:41.881228 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":320,"threads_started":1}
I20260811 12:51:41.883378 9633 heartbeater.cc:347] Connected to a master server at 127.7.111.126:44719
I20260811 12:51:41.883661 9633 heartbeater.cc:464] Registering TS with master...
I20260811 12:51:41.884404 9633 heartbeater.cc:511] Master 127.7.111.126:44719 requested a full tablet report, sending...
I20260811 12:51:41.885757 8847 ts_manager.cc:194] Registered new tserver with Master: 5ab4dc0ae95a4c0886b24a4b4f6fdeac (127.7.111.67:33479)
I20260811 12:51:41.886507 8847 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.111.67:39783
I20260811 12:51:41.897363 7613 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:51:41.906824 9190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:41.901284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48968 (local address 127.7.111.126:44719)
0811 12:51:41.901542 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:41.901549 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:41.901941 (+ 392us) server_negotiation.cc:407] Connection header received
0811 12:51:41.902046 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:41.902050 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:41.902133 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:41.902229 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:41.902625 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:41.903355 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:41.904589 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:41.905030 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:41.905136 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:41.906185 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:41.906190 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:41.906193 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:41.906211 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:41.906244 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:41.906256 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:41.906325 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:41.906415 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 12:51:41.906492 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:51:41.910542 8847 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48968:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:51:41.917132 8847 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:51:41.937539 9652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:41.928579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:33479 (local address 127.0.0.1:50758)
0811 12:51:41.929287 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:41.929323 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:41.929509 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:41.930492 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:41.930499 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:41.930522 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:41.930841 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:41.930855 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:41.932863 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:41.932870 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:41.935154 (+ 2284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:41.935166 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:41.936554 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:41.936561 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:41.936767 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:41.936828 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 12:51:41.936945 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 12:51:41.937140 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"mutex_wait_us":25,"thread_start_us":333,"threads_started":1}
I20260811 12:51:41.938230 9655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:41.928755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50758 (local address 127.7.111.67:33479)
0811 12:51:41.929344 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:41.929350 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:41.929414 (+ 64us) server_negotiation.cc:407] Connection header received
0811 12:51:41.929715 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:41.929723 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:41.929968 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:41.930161 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:41.931004 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:41.932592 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:41.935335 (+ 2743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:41.936325 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:41.936462 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:41.936750 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:41.937707 (+ 957us) server_negotiation.cc:299] Negotiation successful
0811 12:51:41.937882 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":283,"threads_started":1}
I20260811 12:51:41.939365 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:41.929706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:39371 (local address 127.0.0.1:46786)
0811 12:51:41.930213 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:41.930230 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:41.930338 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:41.932281 (+ 1943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:41.932287 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:41.932318 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:41.932923 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:41.932937 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:41.934439 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:41.934445 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:41.936556 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:41.936569 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:41.938588 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:41.938592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:41.938734 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:41.938759 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 12:51:41.938809 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 12:51:41.938895 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":6,"thread_start_us":241,"threads_started":1}
I20260811 12:51:41.940382 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:41.930012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46786 (local address 127.7.111.65:39371)
0811 12:51:41.931740 (+ 1728us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:41.931748 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:41.931775 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:51:41.931869 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:41.931876 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:41.932079 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:41.932260 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:41.933157 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:41.934303 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:41.936855 (+ 2552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:41.938329 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:41.939561 (+ 1232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:41.939808 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:41.939880 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 12:51:41.940046 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1568,"thread_start_us":1463,"threads_started":1}
I20260811 12:51:41.941854 9661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:41.931015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:39703 (local address 127.0.0.1:39048)
0811 12:51:41.933044 (+ 2029us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:41.933062 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:41.933282 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:41.934120 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:41.934125 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:41.934150 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:41.934532 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:41.934543 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:41.936399 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:41.936405 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:41.939011 (+ 2606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:41.939073 (+ 62us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:41.941108 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:41.941114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:41.941296 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:41.941329 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:51:41.941448 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 12:51:41.941568 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1870,"thread_start_us":1574,"threads_started":1}
I20260811 12:51:41.941984 9659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:41.930678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39048 (local address 127.7.111.66:39703)
0811 12:51:41.931341 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:41.931350 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:41.933172 (+ 1822us) server_negotiation.cc:407] Connection header received
0811 12:51:41.933502 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:41.933510 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:41.933726 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:41.933877 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:41.934726 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:41.936257 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:41.939295 (+ 3038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:41.940546 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:41.940691 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:41.940929 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:41.941499 (+ 570us) server_negotiation.cc:299] Negotiation successful
0811 12:51:41.941669 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":409,"threads_started":1}
I20260811 12:51:41.942611 9507 tablet_service.cc:1467] Processing CreateTablet for tablet f293ddb99cc04b5bb03f0d70adb5e1f0 (DEFAULT_TABLE table=test-table [id=5571e6a7250e4d4dad9ab1ad85f15556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:51:41.944228 9507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f293ddb99cc04b5bb03f0d70adb5e1f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:41.944911 9063 tablet_service.cc:1467] Processing CreateTablet for tablet f293ddb99cc04b5bb03f0d70adb5e1f0 (DEFAULT_TABLE table=test-table [id=5571e6a7250e4d4dad9ab1ad85f15556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:51:41.944994 9291 tablet_service.cc:1467] Processing CreateTablet for tablet f293ddb99cc04b5bb03f0d70adb5e1f0 (DEFAULT_TABLE table=test-table [id=5571e6a7250e4d4dad9ab1ad85f15556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:51:41.946202 9291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f293ddb99cc04b5bb03f0d70adb5e1f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:41.946281 9063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f293ddb99cc04b5bb03f0d70adb5e1f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:41.963405 9664 tablet_bootstrap.cc:492] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e: Bootstrap starting.
I20260811 12:51:41.964721 9665 tablet_bootstrap.cc:492] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 5ab4dc0ae95a4c0886b24a4b4f6fdeac: Bootstrap starting.
I20260811 12:51:41.967410 9667 tablet_bootstrap.cc:492] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b: Bootstrap starting.
I20260811 12:51:41.969444 9664 tablet_bootstrap.cc:654] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:41.969643 9665 tablet_bootstrap.cc:654] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 5ab4dc0ae95a4c0886b24a4b4f6fdeac: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:41.970651 9665 log.cc:824] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 5ab4dc0ae95a4c0886b24a4b4f6fdeac: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:41.970817 9667 tablet_bootstrap.cc:654] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:41.970924 9664 log.cc:824] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:41.971937 9667 log.cc:824] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:41.974001 9664 tablet_bootstrap.cc:492] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e: No bootstrap required, opened a new log
I20260811 12:51:41.974237 9664 ts_tablet_manager.cc:1397] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260811 12:51:41.974713 9667 tablet_bootstrap.cc:492] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b: No bootstrap required, opened a new log
I20260811 12:51:41.974970 9667 ts_tablet_manager.cc:1397] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 12:51:41.975063 9665 tablet_bootstrap.cc:492] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 5ab4dc0ae95a4c0886b24a4b4f6fdeac: No bootstrap required, opened a new log
I20260811 12:51:41.975350 9665 ts_tablet_manager.cc:1397] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 5ab4dc0ae95a4c0886b24a4b4f6fdeac: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.007s
I20260811 12:51:41.979154 9188 heartbeater.cc:503] Master 127.7.111.126:44719 was elected leader, sending a full tablet report...
I20260811 12:51:41.982610 9664 raft_consensus.cc:348] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab4dc0ae95a4c0886b24a4b4f6fdeac" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 33479 } } peers { permanent_uuid: "9c15ff0dff734f3589035d74cf01060b" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 39371 } } peers { permanent_uuid: "9d21a743d35345618ba961f8dae9400e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 39703 } }
I20260811 12:51:41.983202 9664 raft_consensus.cc:374] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:41.983361 9664 raft_consensus.cc:729] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d21a743d35345618ba961f8dae9400e, State: Initialized, Role: FOLLOWER
I20260811 12:51:41.984077 9664 consensus_queue.cc:260] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab4dc0ae95a4c0886b24a4b4f6fdeac" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 33479 } } peers { permanent_uuid: "9c15ff0dff734f3589035d74cf01060b" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 39371 } } peers { permanent_uuid: "9d21a743d35345618ba961f8dae9400e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 39703 } }
I20260811 12:51:41.985119 9406 heartbeater.cc:503] Master 127.7.111.126:44719 was elected leader, sending a full tablet report...
I20260811 12:51:41.985312 9667 raft_consensus.cc:348] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab4dc0ae95a4c0886b24a4b4f6fdeac" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 33479 } } peers { permanent_uuid: "9c15ff0dff734f3589035d74cf01060b" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 39371 } } peers { permanent_uuid: "9d21a743d35345618ba961f8dae9400e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 39703 } }
I20260811 12:51:41.985312 9665 raft_consensus.cc:348] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 5ab4dc0ae95a4c0886b24a4b4f6fdeac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab4dc0ae95a4c0886b24a4b4f6fdeac" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 33479 } } peers { permanent_uuid: "9c15ff0dff734f3589035d74cf01060b" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 39371 } } peers { permanent_uuid: "9d21a743d35345618ba961f8dae9400e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 39703 } }
I20260811 12:51:41.985742 9667 raft_consensus.cc:374] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:41.985743 9665 raft_consensus.cc:374] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 5ab4dc0ae95a4c0886b24a4b4f6fdeac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:41.985826 9664 ts_tablet_manager.cc:1428] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e: Time spent starting tablet: real 0.011s user 0.010s sys 0.001s
I20260811 12:51:41.985833 9667 raft_consensus.cc:729] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c15ff0dff734f3589035d74cf01060b, State: Initialized, Role: FOLLOWER
I20260811 12:51:41.985838 9665 raft_consensus.cc:729] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 5ab4dc0ae95a4c0886b24a4b4f6fdeac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab4dc0ae95a4c0886b24a4b4f6fdeac, State: Initialized, Role: FOLLOWER
I20260811 12:51:41.986408 9665 consensus_queue.cc:260] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 5ab4dc0ae95a4c0886b24a4b4f6fdeac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab4dc0ae95a4c0886b24a4b4f6fdeac" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 33479 } } peers { permanent_uuid: "9c15ff0dff734f3589035d74cf01060b" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 39371 } } peers { permanent_uuid: "9d21a743d35345618ba961f8dae9400e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 39703 } }
I20260811 12:51:41.986459 9667 consensus_queue.cc:260] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab4dc0ae95a4c0886b24a4b4f6fdeac" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 33479 } } peers { permanent_uuid: "9c15ff0dff734f3589035d74cf01060b" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 39371 } } peers { permanent_uuid: "9d21a743d35345618ba961f8dae9400e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 39703 } }
I20260811 12:51:41.988097 9665 ts_tablet_manager.cc:1428] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 5ab4dc0ae95a4c0886b24a4b4f6fdeac: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260811 12:51:41.988495 9667 ts_tablet_manager.cc:1428] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260811 12:51:41.988682 9633 heartbeater.cc:503] Master 127.7.111.126:44719 was elected leader, sending a full tablet report...
I20260811 12:51:42.029361 9671 raft_consensus.cc:482] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:51:42.029675 9671 raft_consensus.cc:504] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab4dc0ae95a4c0886b24a4b4f6fdeac" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 33479 } } peers { permanent_uuid: "9c15ff0dff734f3589035d74cf01060b" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 39371 } } peers { permanent_uuid: "9d21a743d35345618ba961f8dae9400e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 39703 } }
I20260811 12:51:42.031368 9671 leader_election.cc:288] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ab4dc0ae95a4c0886b24a4b4f6fdeac (127.7.111.67:33479), 9c15ff0dff734f3589035d74cf01060b (127.7.111.65:39371)
I20260811 12:51:42.032853 9672 raft_consensus.cc:482] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:51:42.033183 9672 raft_consensus.cc:504] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab4dc0ae95a4c0886b24a4b4f6fdeac" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 33479 } } peers { permanent_uuid: "9c15ff0dff734f3589035d74cf01060b" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 39371 } } peers { permanent_uuid: "9d21a743d35345618ba961f8dae9400e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 39703 } }
I20260811 12:51:42.035233 9672 leader_election.cc:288] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ab4dc0ae95a4c0886b24a4b4f6fdeac (127.7.111.67:33479), 9d21a743d35345618ba961f8dae9400e (127.7.111.66:39703)
I20260811 12:51:42.040369 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:42.032768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:46121 (local address 127.7.111.65:39371)
0811 12:51:42.032971 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:42.032977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:42.033726 (+ 749us) server_negotiation.cc:407] Connection header received
0811 12:51:42.033952 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:42.033956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:42.034031 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:42.034113 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:42.034614 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.035554 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.037834 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.039027 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.039154 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:42.039232 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:42.039757 (+ 525us) server_negotiation.cc:299] Negotiation successful
0811 12:51:42.039960 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:51:42.040956 9655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:42.031789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.66:50793 (local address 127.7.111.67:33479)
0811 12:51:42.032002 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:42.032010 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:42.032509 (+ 499us) server_negotiation.cc:407] Connection header received
0811 12:51:42.032827 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:42.032833 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:42.032913 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:42.032977 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:42.033853 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.034888 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.038599 (+ 3711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.039851 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.039970 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:42.040032 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:42.040542 (+ 510us) server_negotiation.cc:299] Negotiation successful
0811 12:51:42.040693 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:51:42.041270 9680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:42.032435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.65:39371 (local address 127.7.111.66:46121)
0811 12:51:42.033609 (+ 1174us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:42.033629 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:42.033768 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:42.034141 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:42.034146 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:42.034169 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:42.034459 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:42.034468 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.035724 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.035729 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:42.037609 (+ 1880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:42.037623 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.039394 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.039400 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:42.039563 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:42.039595 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:51:42.040786 (+ 1191us) client_negotiation.cc:240] Negotiation successful
0811 12:51:42.040967 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1054,"thread_start_us":821,"threads_started":1}
I20260811 12:51:42.041996 9679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:42.031727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:33479 (local address 127.7.111.66:50793)
0811 12:51:42.032372 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:42.032399 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:42.032648 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:42.033180 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:42.033186 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:42.033230 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:42.033715 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:42.033728 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.035026 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.035032 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:42.038387 (+ 3355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:42.038401 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.040190 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.040196 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:42.040353 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:42.040385 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:51:42.041580 (+ 1195us) client_negotiation.cc:240] Negotiation successful
0811 12:51:42.041715 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":334,"threads_started":1}
I20260811 12:51:42.042374 9105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f293ddb99cc04b5bb03f0d70adb5e1f0" candidate_uuid: "9d21a743d35345618ba961f8dae9400e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c15ff0dff734f3589035d74cf01060b" is_pre_election: true
I20260811 12:51:42.042900 9105 raft_consensus.cc:2454] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d21a743d35345618ba961f8dae9400e in term 0.
I20260811 12:51:42.043229 9543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f293ddb99cc04b5bb03f0d70adb5e1f0" candidate_uuid: "9d21a743d35345618ba961f8dae9400e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab4dc0ae95a4c0886b24a4b4f6fdeac" is_pre_election: true
I20260811 12:51:42.043674 9222 leader_election.cc:302] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c15ff0dff734f3589035d74cf01060b, 9d21a743d35345618ba961f8dae9400e; no voters:
I20260811 12:51:42.043826 9543 raft_consensus.cc:2454] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 5ab4dc0ae95a4c0886b24a4b4f6fdeac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d21a743d35345618ba961f8dae9400e in term 0.
I20260811 12:51:42.044207 9671 raft_consensus.cc:2781] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:51:42.044884 9671 raft_consensus.cc:482] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:51:42.045018 9671 raft_consensus.cc:3037] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:42.047771 9671 raft_consensus.cc:504] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab4dc0ae95a4c0886b24a4b4f6fdeac" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 33479 } } peers { permanent_uuid: "9c15ff0dff734f3589035d74cf01060b" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 39371 } } peers { permanent_uuid: "9d21a743d35345618ba961f8dae9400e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 39703 } }
I20260811 12:51:42.048827 9543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f293ddb99cc04b5bb03f0d70adb5e1f0" candidate_uuid: "9d21a743d35345618ba961f8dae9400e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab4dc0ae95a4c0886b24a4b4f6fdeac"
I20260811 12:51:42.049038 9671 leader_election.cc:288] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e [CANDIDATE]: Term 1 election: Requested vote from peers 5ab4dc0ae95a4c0886b24a4b4f6fdeac (127.7.111.67:33479), 9c15ff0dff734f3589035d74cf01060b (127.7.111.65:39371)
I20260811 12:51:42.049072 9543 raft_consensus.cc:3037] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 5ab4dc0ae95a4c0886b24a4b4f6fdeac [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:42.049280 9682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:42.035473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.67:33479 (local address 127.7.111.65:53305)
0811 12:51:42.042635 (+ 7162us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:42.042656 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:42.042826 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:42.043330 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:42.043336 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:42.043362 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:42.043758 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:42.043772 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.045138 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.045144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:42.047196 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:42.047209 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.048362 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.048368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:42.048524 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:42.048548 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 12:51:42.048633 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 12:51:42.048773 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7029,"mutex_wait_us":1,"thread_start_us":6801,"threads_started":1}
I20260811 12:51:42.049685 9105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f293ddb99cc04b5bb03f0d70adb5e1f0" candidate_uuid: "9d21a743d35345618ba961f8dae9400e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c15ff0dff734f3589035d74cf01060b"
I20260811 12:51:42.049858 9105 raft_consensus.cc:3037] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:42.051486 9659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:42.043078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:53745 (local address 127.7.111.66:39703)
0811 12:51:42.043272 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:42.043277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:42.044799 (+ 1522us) server_negotiation.cc:407] Connection header received
0811 12:51:42.044962 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:42.044966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:42.045039 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:42.045126 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:42.045915 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.047004 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.049294 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.050425 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.050539 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:42.050598 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:42.051137 (+ 539us) server_negotiation.cc:299] Negotiation successful
0811 12:51:42.051267 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 12:51:42.051508 9685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:42.043565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.66:39703 (local address 127.7.111.65:53745)
0811 12:51:42.044701 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0811 12:51:42.044715 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:51:42.044829 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:51:42.045212 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:51:42.045218 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:51:42.045252 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:51:42.045739 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:42.045757 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.047163 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.047169 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:42.049038 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:51:42.049053 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.050780 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.050787 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:51:42.050960 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:42.050995 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 12:51:42.051087 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 12:51:42.051222 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1023,"thread_start_us":847,"threads_started":1}
I20260811 12:51:42.052191 9543 raft_consensus.cc:2454] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 5ab4dc0ae95a4c0886b24a4b4f6fdeac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d21a743d35345618ba961f8dae9400e in term 1.
I20260811 12:51:42.052505 9222 leader_election.cc:302] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab4dc0ae95a4c0886b24a4b4f6fdeac, 9d21a743d35345618ba961f8dae9400e; no voters:
I20260811 12:51:42.052491 9105 raft_consensus.cc:2454] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d21a743d35345618ba961f8dae9400e in term 1.
I20260811 12:51:42.052763 9671 raft_consensus.cc:2781] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:51:42.052946 9655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:42.035789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.111.65:53305 (local address 127.7.111.67:33479)
0811 12:51:42.041540 (+ 5751us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:42.041544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:42.042725 (+ 1181us) server_negotiation.cc:407] Connection header received
0811 12:51:42.043007 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:42.043012 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:42.043081 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:42.043139 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:51:42.043924 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.044960 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.047378 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.048189 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.048337 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:42.048402 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:42.048672 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 12:51:42.048797 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5487,"thread_start_us":6976,"threads_started":1}
I20260811 12:51:42.053493 9671 raft_consensus.cc:686] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e [term 1 LEADER]: Becoming Leader. State: Replica: 9d21a743d35345618ba961f8dae9400e, State: Running, Role: LEADER
I20260811 12:51:42.053659 9543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f293ddb99cc04b5bb03f0d70adb5e1f0" candidate_uuid: "9c15ff0dff734f3589035d74cf01060b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab4dc0ae95a4c0886b24a4b4f6fdeac" is_pre_election: true
I20260811 12:51:42.053632 9325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f293ddb99cc04b5bb03f0d70adb5e1f0" candidate_uuid: "9c15ff0dff734f3589035d74cf01060b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d21a743d35345618ba961f8dae9400e" is_pre_election: true
I20260811 12:51:42.053963 9543 raft_consensus.cc:2379] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 5ab4dc0ae95a4c0886b24a4b4f6fdeac [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c15ff0dff734f3589035d74cf01060b in current term 1: Already voted for candidate 9d21a743d35345618ba961f8dae9400e in this term.
I20260811 12:51:42.054224 9671 consensus_queue.cc:237] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab4dc0ae95a4c0886b24a4b4f6fdeac" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 33479 } } peers { permanent_uuid: "9c15ff0dff734f3589035d74cf01060b" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 39371 } } peers { permanent_uuid: "9d21a743d35345618ba961f8dae9400e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 39703 } }
I20260811 12:51:42.054862 9000 leader_election.cc:302] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c15ff0dff734f3589035d74cf01060b; no voters: 5ab4dc0ae95a4c0886b24a4b4f6fdeac, 9d21a743d35345618ba961f8dae9400e
I20260811 12:51:42.055234 9672 raft_consensus.cc:2726] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 12:51:42.058756 8842 catalog_manager.cc:5495] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d21a743d35345618ba961f8dae9400e (127.7.111.66). New cstate: current_term: 1 leader_uuid: "9d21a743d35345618ba961f8dae9400e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab4dc0ae95a4c0886b24a4b4f6fdeac" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 33479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c15ff0dff734f3589035d74cf01060b" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 39371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d21a743d35345618ba961f8dae9400e" member_type: VOTER last_known_addr { host: "127.7.111.66" port: 39703 } health_report { overall_health: HEALTHY } } }
I20260811 12:51:42.103124 9659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:42.097936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39064 (local address 127.7.111.66:39703)
0811 12:51:42.098160 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:42.098166 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:42.098221 (+ 55us) server_negotiation.cc:407] Connection header received
0811 12:51:42.098401 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:42.098405 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:42.098505 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:42.098676 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:51:42.098980 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.099800 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.101137 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.101587 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.101678 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:42.102136 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:51:42.102637 (+ 501us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:51:42.102764 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:42.102806 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 12:51:42.102871 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 12:51:42.111305 9543 raft_consensus.cc:1261] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 5ab4dc0ae95a4c0886b24a4b4f6fdeac [term 1 FOLLOWER]: Refusing update from remote peer 9d21a743d35345618ba961f8dae9400e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:51:42.111352 9105 raft_consensus.cc:1261] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b [term 1 FOLLOWER]: Refusing update from remote peer 9d21a743d35345618ba961f8dae9400e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:51:42.112193 9688 consensus_queue.cc:1035] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c15ff0dff734f3589035d74cf01060b" member_type: VOTER last_known_addr { host: "127.7.111.65" port: 39371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:51:42.112528 9671 consensus_queue.cc:1035] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab4dc0ae95a4c0886b24a4b4f6fdeac" member_type: VOTER last_known_addr { host: "127.7.111.67" port: 33479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 12:51:42.120355 9634 tablet.cc:2369] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 5ab4dc0ae95a4c0886b24a4b4f6fdeac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:51:42.139814 8847 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:51:42.140113 8847 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:51:42.143942 8847 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e77597fad124e82b082e61d72779b8f: Sending DeleteTablet for 3 replicas of tablet f293ddb99cc04b5bb03f0d70adb5e1f0
I20260811 12:51:42.145083 9507 tablet_service.cc:1514] Processing DeleteTablet for tablet f293ddb99cc04b5bb03f0d70adb5e1f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:51:42 UTC) from {username='slave'} at 127.0.0.1:50758
I20260811 12:51:42.145295 9291 tablet_service.cc:1514] Processing DeleteTablet for tablet f293ddb99cc04b5bb03f0d70adb5e1f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:51:42 UTC) from {username='slave'} at 127.0.0.1:39048
I20260811 12:51:42.145896 9063 tablet_service.cc:1514] Processing DeleteTablet for tablet f293ddb99cc04b5bb03f0d70adb5e1f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:51:42 UTC) from {username='slave'} at 127.0.0.1:46786
I20260811 12:51:42.146442 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 8969
I20260811 12:51:42.154803 9709 tablet_replica.cc:330] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b: stopping tablet replica
I20260811 12:51:42.155305 9709 raft_consensus.cc:2229] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:51:42.155901 9709 raft_consensus.cc:2258] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:51:42.156757 9705 tablet_replica.cc:330] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e: stopping tablet replica
I20260811 12:51:42.157166 9705 raft_consensus.cc:2229] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:51:42.158015 9705 raft_consensus.cc:2258] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:51:42.158215 9709 ts_tablet_manager.cc:1905] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:51:42.160472 9705 ts_tablet_manager.cc:1905] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:51:42.164026 9704 tablet_replica.cc:330] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 5ab4dc0ae95a4c0886b24a4b4f6fdeac: stopping tablet replica
I20260811 12:51:42.164594 9704 raft_consensus.cc:2229] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 5ab4dc0ae95a4c0886b24a4b4f6fdeac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:51:42.165187 9704 raft_consensus.cc:2258] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 5ab4dc0ae95a4c0886b24a4b4f6fdeac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:51:42.165854 9709 ts_tablet_manager.cc:1918] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:51:42.166021 9709 log.cc:1196] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-0/wal/wals/f293ddb99cc04b5bb03f0d70adb5e1f0
I20260811 12:51:42.166580 9709 ts_tablet_manager.cc:1939] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9c15ff0dff734f3589035d74cf01060b: Deleting consensus metadata
I20260811 12:51:42.168257 9704 ts_tablet_manager.cc:1905] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 5ab4dc0ae95a4c0886b24a4b4f6fdeac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:51:42.168602 8826 catalog_manager.cc:4841] TS 9c15ff0dff734f3589035d74cf01060b (127.7.111.65:39371): tablet f293ddb99cc04b5bb03f0d70adb5e1f0 (table test-table [id=5571e6a7250e4d4dad9ab1ad85f15556]) successfully deleted
I20260811 12:51:42.170902 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:42.147842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46796 (local address 127.7.111.65:39371)
0811 12:51:42.148028 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:42.148035 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:42.153939 (+ 5904us) server_negotiation.cc:407] Connection header received
0811 12:51:42.154141 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:42.154147 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:42.154292 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:42.154390 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:42.155959 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.157173 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.164939 (+ 7766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.165673 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.165783 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:42.169173 (+ 3390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:42.169197 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:42.169219 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:42.170016 (+ 797us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:42.170078 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:42.170105 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:42.170229 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:42.170438 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 12:51:42.170563 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:51:42.172209 9181 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:42.172797 9705 ts_tablet_manager.cc:1918] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:51:42.172976 9705 log.cc:1196] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-1/wal/wals/f293ddb99cc04b5bb03f0d70adb5e1f0
I20260811 12:51:42.173595 9705 ts_tablet_manager.cc:1939] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 9d21a743d35345618ba961f8dae9400e: Deleting consensus metadata
I20260811 12:51:42.175374 8826 catalog_manager.cc:4841] TS 9d21a743d35345618ba961f8dae9400e (127.7.111.66:39703): tablet f293ddb99cc04b5bb03f0d70adb5e1f0 (table test-table [id=5571e6a7250e4d4dad9ab1ad85f15556]) successfully deleted
I20260811 12:51:42.176411 9704 ts_tablet_manager.cc:1918] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 5ab4dc0ae95a4c0886b24a4b4f6fdeac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:51:42.176585 9704 log.cc:1196] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 5ab4dc0ae95a4c0886b24a4b4f6fdeac: Deleting WAL directory at /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786452375851340-7613-0/minicluster-data/ts-2/wal/wals/f293ddb99cc04b5bb03f0d70adb5e1f0
I20260811 12:51:42.177142 9704 ts_tablet_manager.cc:1939] T f293ddb99cc04b5bb03f0d70adb5e1f0 P 5ab4dc0ae95a4c0886b24a4b4f6fdeac: Deleting consensus metadata
I20260811 12:51:42.178990 8826 catalog_manager.cc:4841] TS 5ab4dc0ae95a4c0886b24a4b4f6fdeac (127.7.111.67:33479): tablet f293ddb99cc04b5bb03f0d70adb5e1f0 (table test-table [id=5571e6a7250e4d4dad9ab1ad85f15556]) successfully deleted
I20260811 12:51:42.306315 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 8969
I20260811 12:51:42.332731 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 9192
I20260811 12:51:42.339742 9659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:42.333527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39070 (local address 127.7.111.66:39703)
0811 12:51:42.333722 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:42.333728 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:42.333818 (+ 90us) server_negotiation.cc:407] Connection header received
0811 12:51:42.334021 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:42.334025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:42.334134 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:42.334230 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:42.334652 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.335404 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.336682 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.337109 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.337232 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:42.338330 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:42.338339 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:42.338355 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:42.338997 (+ 642us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:42.339045 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:42.339072 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:42.339188 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:42.339316 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 12:51:42.339417 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 12:51:42.340389 9397 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:42.453059 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 9192
I20260811 12:51:42.480715 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 9409
I20260811 12:51:42.487203 9655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:42.481496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50764 (local address 127.7.111.67:33479)
0811 12:51:42.481691 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:42.481697 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:42.481721 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:51:42.481859 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:42.481864 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:42.481981 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:42.482071 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:42.482409 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.483078 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.484418 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:42.485019 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:42.485119 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:42.486019 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:42.486028 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:42.486045 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:42.486653 (+ 608us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:42.486709 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:42.486730 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:42.486816 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:42.486890 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 12:51:42.486969 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:51:42.487843 9627 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:42.580869 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 9409
I20260811 12:51:42.604334 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 8795
I20260811 12:51:42.605435 8916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:42.714563 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 8795
2026-08-11T12:51:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2902 ms)
[----------] 6 tests from EncryptionPolicyTest (23561 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:51:42 dist-test-slave-f0tn krb5kdc[9724](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:51:42 dist-test-slave-f0tn krb5kdc[9724](info): set up 2 sockets
Aug 11 12:51:42 dist-test-slave-f0tn krb5kdc[9724](info): commencing operation
krb5kdc: starting...
W20260811 12:51:43.986900 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.011s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:51:44 dist-test-slave-f0tn krb5kdc[9724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:51:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:51:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:51:44.340780 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33695
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:51:44.683681 9748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:44.683952 9748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:44.684054 9748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:44.691179 9748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:51:44.691306 9748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:44.691382 9748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:44.691466 9748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:51:44.691529 9748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:51:44.701097 9748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33695
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:44.702436 9748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:44.703764 9748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:44.715282 9758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:44.715314 9759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:44.715440 9764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:44.715966 9748 server_base.cc:1034] running on GCE node
I20260811 12:51:44.716668 9748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:44.718024 9748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:44.719219 9748 hybrid_clock.cc:648] HybridClock initialized: now 1786452704719181 us; error 30 us; skew 500 ppm
I20260811 12:51:44.719527 9748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:51:44 dist-test-slave-f0tn krb5kdc[9724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452704, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:51:44.726127 9748 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:51:44.727512 9748 webserver.cc:460] Webserver started at http://127.0.0.1:34691/ using document root <none> and password file <none>
I20260811 12:51:44.728188 9748 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:44.728305 9748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:44.728600 9748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:44.731557 9748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "a7757888426849ce8091ea91ab88c508"
format_stamp: "Formatted at 2026-08-11 12:51:44 on dist-test-slave-f0tn"
server_key: "0275c14a0ba51d995bc7086fae7ae948"
server_key_iv: "3042c593de87a1c839106d57b72d2f9a"
server_key_version: "encryptionkey@0"
I20260811 12:51:44.732321 9748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "a7757888426849ce8091ea91ab88c508"
format_stamp: "Formatted at 2026-08-11 12:51:44 on dist-test-slave-f0tn"
server_key: "0275c14a0ba51d995bc7086fae7ae948"
server_key_iv: "3042c593de87a1c839106d57b72d2f9a"
server_key_version: "encryptionkey@0"
I20260811 12:51:44.737226 9748 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 12:51:44.740543 9775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:44.741842 9748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:51:44.741993 9748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "a7757888426849ce8091ea91ab88c508"
format_stamp: "Formatted at 2026-08-11 12:51:44 on dist-test-slave-f0tn"
server_key: "0275c14a0ba51d995bc7086fae7ae948"
server_key_iv: "3042c593de87a1c839106d57b72d2f9a"
server_key_version: "encryptionkey@0"
I20260811 12:51:44.742163 9748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:44.763115 9748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:44.766959 9748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:44.767232 9748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:44.784605 9748 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33695
I20260811 12:51:44.784634 9861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33695 every 8 connection(s)
I20260811 12:51:44.785952 9748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:51:44.789644 9863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:44.792142 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 9748
I20260811 12:51:44.792368 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:51:44.793192 7613 external_mini_cluster.cc:1428] Setting key 285feb60218f37b371ed22458450c362
I20260811 12:51:44.801769 9863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7757888426849ce8091ea91ab88c508: Bootstrap starting.
I20260811 12:51:44.805446 9863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7757888426849ce8091ea91ab88c508: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:44.806754 9863 log.cc:824] T 00000000000000000000000000000000 P a7757888426849ce8091ea91ab88c508: Log is configured to *not* fsync() on all Append() calls
Aug 11 12:51:44 dist-test-slave-f0tn krb5kdc[9724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 12:51:44.810050 9863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7757888426849ce8091ea91ab88c508: No bootstrap required, opened a new log
I20260811 12:51:44.817572 9874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:44.795223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48011 (local address 127.0.0.1:33695)
0811 12:51:44.796008 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:44.796020 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:44.796061 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:51:44.797710 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:44.797732 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:44.798163 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:44.798481 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:44.800189 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:44.801491 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:44.803197 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:44.803897 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:44.804023 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:44.809034 (+ 5011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:44.809046 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:44.809067 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:44.809111 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:44.812895 (+ 3784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:44.813771 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:44.813779 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:44.813788 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:44.814005 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:44.814448 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:44.814454 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:44.814458 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:44.815134 (+ 676us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:44.815476 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:44.815788 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 12:51:44.816132 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":351,"threads_started":1}
I20260811 12:51:44.819854 9863 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7757888426849ce8091ea91ab88c508 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7757888426849ce8091ea91ab88c508" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33695 } }
I20260811 12:51:44.820245 9863 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7757888426849ce8091ea91ab88c508 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:44.820349 9863 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7757888426849ce8091ea91ab88c508 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7757888426849ce8091ea91ab88c508, State: Initialized, Role: FOLLOWER
I20260811 12:51:44.820945 9863 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7757888426849ce8091ea91ab88c508 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7757888426849ce8091ea91ab88c508" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33695 } }
I20260811 12:51:44.821183 9863 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7757888426849ce8091ea91ab88c508 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:51:44.821311 9863 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7757888426849ce8091ea91ab88c508 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:51:44.821507 9863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7757888426849ce8091ea91ab88c508 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:44.821650 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:51:44.824693 9863 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7757888426849ce8091ea91ab88c508 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7757888426849ce8091ea91ab88c508" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33695 } }
I20260811 12:51:44.825305 9863 leader_election.cc:302] T 00000000000000000000000000000000 P a7757888426849ce8091ea91ab88c508 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7757888426849ce8091ea91ab88c508; no voters:
I20260811 12:51:44.826035 9863 leader_election.cc:288] T 00000000000000000000000000000000 P a7757888426849ce8091ea91ab88c508 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:51:44.826222 9884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7757888426849ce8091ea91ab88c508 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:51:44.829406 9863 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7757888426849ce8091ea91ab88c508 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:51:44.829658 9884 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7757888426849ce8091ea91ab88c508 [term 1 LEADER]: Becoming Leader. State: Replica: a7757888426849ce8091ea91ab88c508, State: Running, Role: LEADER
I20260811 12:51:44.830220 9884 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7757888426849ce8091ea91ab88c508 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7757888426849ce8091ea91ab88c508" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33695 } }
I20260811 12:51:44.832790 9890 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7757888426849ce8091ea91ab88c508 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7757888426849ce8091ea91ab88c508" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7757888426849ce8091ea91ab88c508" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33695 } } }
I20260811 12:51:44.833168 9890 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7757888426849ce8091ea91ab88c508 [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:44.833668 9891 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7757888426849ce8091ea91ab88c508 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7757888426849ce8091ea91ab88c508. Latest consensus state: current_term: 1 leader_uuid: "a7757888426849ce8091ea91ab88c508" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7757888426849ce8091ea91ab88c508" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33695 } } }
I20260811 12:51:44.833994 9891 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7757888426849ce8091ea91ab88c508 [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:44.837574 9897 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:51:44.843346 9897 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:51:44.857734 9874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:44.827915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39515 (local address 127.0.0.1:33695)
0811 12:51:44.833515 (+ 5600us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:44.833522 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:44.838048 (+ 4526us) server_negotiation.cc:407] Connection header received
0811 12:51:44.838237 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:44.838243 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:44.838335 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:44.838463 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:44.840732 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:44.842092 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:44.844110 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:44.844938 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:44.845100 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:44.849049 (+ 3949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:44.849057 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:44.849062 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:44.849109 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:44.853170 (+ 4061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:44.854240 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:44.854247 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:44.854251 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:44.854413 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:44.854987 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:44.854992 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:44.854996 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:44.855444 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:44.855735 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:44.857106 (+ 1371us) server_negotiation.cc:299] Negotiation successful
0811 12:51:44.857311 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5439}
I20260811 12:51:44.861297 9897 catalog_manager.cc:1347] Generated new cluster ID: 75f227a73fa74711b454981315165355
I20260811 12:51:44.861438 9897 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:51:44.872128 9897 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:51:44.873903 9897 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:51:44.889515 9897 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7757888426849ce8091ea91ab88c508: Generated new TSK 0
I20260811 12:51:44.890583 9897 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:51:44.905242 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 9748
I20260811 12:51:44.907251 9856 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:44.998267 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 9748
W20260811 12:51:45.021546 9721 connection.cc:535] client connection to 127.0.0.1:33695 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33695): Connection reset by peer (error 104)
2026-08-11T12:51:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2271 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:51:45 dist-test-slave-f0tn krb5kdc[9920](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:51:45 dist-test-slave-f0tn krb5kdc[9920](info): set up 2 sockets
Aug 11 12:51:45 dist-test-slave-f0tn krb5kdc[9920](info): commencing operation
krb5kdc: starting...
W20260811 12:51:46.271023 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.001s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:51:46 dist-test-slave-f0tn krb5kdc[9920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:51:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:51:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:51:46.605922 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38991
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38991
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:51:46.940786 9941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:46.941078 9941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:46.941220 9941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:46.948345 9941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:51:46.948457 9941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:46.948536 9941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:46.948619 9941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:51:46.948679 9941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:51:46.958160 9941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38991
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:46.959374 9941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:46.960654 9941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:46.971827 9952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:46.971865 9953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:46.971890 9955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:46.972841 9941 server_base.cc:1034] running on GCE node
I20260811 12:51:46.973565 9941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:46.974798 9941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:46.976003 9941 hybrid_clock.cc:648] HybridClock initialized: now 1786452706975952 us; error 41 us; skew 500 ppm
I20260811 12:51:46.976321 9941 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:51:46 dist-test-slave-f0tn krb5kdc[9920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452706, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:51:46.979197 9941 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:51:46.980564 9941 webserver.cc:460] Webserver started at http://127.0.0.1:45337/ using document root <none> and password file <none>
I20260811 12:51:46.981218 9941 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:46.981338 9941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:46.981650 9941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:46.984505 9941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "30453229caae455fae03666194698d74"
format_stamp: "Formatted at 2026-08-11 12:51:46 on dist-test-slave-f0tn"
server_key: "3177df86d7681588c7033cfb63b00f2c"
server_key_iv: "3197f1ce0587b53c802520a8d205c8f9"
server_key_version: "encryptionkey@0"
I20260811 12:51:46.985232 9941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "30453229caae455fae03666194698d74"
format_stamp: "Formatted at 2026-08-11 12:51:46 on dist-test-slave-f0tn"
server_key: "3177df86d7681588c7033cfb63b00f2c"
server_key_iv: "3197f1ce0587b53c802520a8d205c8f9"
server_key_version: "encryptionkey@0"
I20260811 12:51:46.990253 9941 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 12:51:46.993575 9966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:46.994837 9941 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:51:46.995010 9941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "30453229caae455fae03666194698d74"
format_stamp: "Formatted at 2026-08-11 12:51:46 on dist-test-slave-f0tn"
server_key: "3177df86d7681588c7033cfb63b00f2c"
server_key_iv: "3197f1ce0587b53c802520a8d205c8f9"
server_key_version: "encryptionkey@0"
I20260811 12:51:46.995177 9941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:47.007927 9941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:47.011554 9941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:47.011799 9941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:47.028553 9941 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38991
I20260811 12:51:47.028592 10042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38991 every 8 connection(s)
I20260811 12:51:47.029874 9941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:51:47.033514 10044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:47.035167 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 9941
I20260811 12:51:47.035326 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:51:47.035750 7613 external_mini_cluster.cc:1428] Setting key 1b5df5acfd423fa2ed2916d1499a2506
I20260811 12:51:47.048069 10044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30453229caae455fae03666194698d74: Bootstrap starting.
I20260811 12:51:47.050057 10048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:47.037468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46783 (local address 127.0.0.1:38991)
0811 12:51:47.038321 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:47.038336 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:47.038386 (+ 50us) server_negotiation.cc:407] Connection header received
0811 12:51:47.040128 (+ 1742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:47.040163 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:47.040697 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:47.041180 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:47.041703 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:47.043567 (+ 1864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:47.044808 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:47.045918 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:47.046096 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:47.047238 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:47.047250 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:47.047269 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:47.047982 (+ 713us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:47.048093 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:47.048128 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:47.048286 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:47.048533 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 12:51:47.048725 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":350,"threads_started":1}
I20260811 12:51:47.051752 10044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30453229caae455fae03666194698d74: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:47.052894 10044 log.cc:824] T 00000000000000000000000000000000 P 30453229caae455fae03666194698d74: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:47.053936 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:51:47.055686 10044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30453229caae455fae03666194698d74: No bootstrap required, opened a new log
I20260811 12:51:47.064749 10048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:47.058219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48599 (local address 127.0.0.1:38991)
0811 12:51:47.058408 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:47.058414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:47.058764 (+ 350us) server_negotiation.cc:407] Connection header received
0811 12:51:47.058925 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:47.058934 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:47.059013 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:47.059115 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:47.059563 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:47.060692 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:47.062099 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:47.062824 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:47.062967 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:47.064038 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:47.064045 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:47.064049 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:47.064070 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:47.064115 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:47.064135 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:47.064251 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:47.064368 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 12:51:47.064422 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:51:47.064703 10044 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30453229caae455fae03666194698d74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30453229caae455fae03666194698d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38991 } }
I20260811 12:51:47.065171 10044 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30453229caae455fae03666194698d74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:47.065285 10044 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30453229caae455fae03666194698d74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30453229caae455fae03666194698d74, State: Initialized, Role: FOLLOWER
I20260811 12:51:47.065907 10044 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30453229caae455fae03666194698d74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30453229caae455fae03666194698d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38991 } }
I20260811 12:51:47.066150 10044 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30453229caae455fae03666194698d74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:51:47.066303 10044 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30453229caae455fae03666194698d74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:51:47.066488 10044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30453229caae455fae03666194698d74 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:47.068994 10044 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30453229caae455fae03666194698d74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30453229caae455fae03666194698d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38991 } }
I20260811 12:51:47.069530 10044 leader_election.cc:302] T 00000000000000000000000000000000 P 30453229caae455fae03666194698d74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30453229caae455fae03666194698d74; no voters:
I20260811 12:51:47.070103 10044 leader_election.cc:288] T 00000000000000000000000000000000 P 30453229caae455fae03666194698d74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:51:47.070237 10059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30453229caae455fae03666194698d74 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:51:47.071269 10059 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30453229caae455fae03666194698d74 [term 1 LEADER]: Becoming Leader. State: Replica: 30453229caae455fae03666194698d74, State: Running, Role: LEADER
I20260811 12:51:47.071808 10059 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30453229caae455fae03666194698d74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30453229caae455fae03666194698d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38991 } }
I20260811 12:51:47.072211 10044 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30453229caae455fae03666194698d74 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:51:47.074795 10068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30453229caae455fae03666194698d74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30453229caae455fae03666194698d74. Latest consensus state: current_term: 1 leader_uuid: "30453229caae455fae03666194698d74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30453229caae455fae03666194698d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38991 } } }
I20260811 12:51:47.074887 10065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30453229caae455fae03666194698d74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30453229caae455fae03666194698d74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30453229caae455fae03666194698d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38991 } } }
I20260811 12:51:47.075186 10068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30453229caae455fae03666194698d74 [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:47.075233 10065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30453229caae455fae03666194698d74 [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:47.076359 10084 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:51:47.081913 10084 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:51:47.090569 10084 catalog_manager.cc:1347] Generated new cluster ID: 59b36a3a0443462dadb5e816b3185b09
I20260811 12:51:47.090699 10084 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:51:47.125983 10084 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:51:47.127360 10084 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:51:47.137354 10084 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30453229caae455fae03666194698d74: Generated new TSK 0
I20260811 12:51:47.138281 10084 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:51:47.147765 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 9941
W20260811 12:51:47.149655 10038 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46783
W20260811 12:51:47.150460 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 9941: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:51:47.150655 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 9941
W20260811 12:51:47.179085 9918 connection.cc:535] client connection to 127.0.0.1:38991 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38991): Connection reset by peer (error 104)
2026-08-11T12:51:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2157 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:51:47 dist-test-slave-f0tn krb5kdc[10099](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:51:47 dist-test-slave-f0tn krb5kdc[10099](info): set up 2 sockets
Aug 11 12:51:47 dist-test-slave-f0tn krb5kdc[10099](info): commencing operation
krb5kdc: starting...
W20260811 12:51:48.422698 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.000s sys 0.058s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:51:48 dist-test-slave-f0tn krb5kdc[10099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:51:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:51:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:51:48.765367 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41261
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:51:49.109509 10116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:49.109869 10116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:49.110049 10116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:49.117475 10116 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:51:49.117619 10116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:49.117710 10116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:49.117807 10116 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:51:49.117879 10116 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:51:49.128265 10116 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41261
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:49.129727 10116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:49.131178 10116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:49.143419 10130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:49.143639 10132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:49.143438 10126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:49.144495 10116 server_base.cc:1034] running on GCE node
I20260811 12:51:49.145334 10116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:49.146883 10116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:49.148100 10116 hybrid_clock.cc:648] HybridClock initialized: now 1786452709148044 us; error 47 us; skew 500 ppm
I20260811 12:51:49.148425 10116 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:51:49 dist-test-slave-f0tn krb5kdc[10099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452709, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:51:49.151613 10116 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:51:49.153034 10116 webserver.cc:460] Webserver started at http://127.0.0.1:32999/ using document root <none> and password file <none>
I20260811 12:51:49.153753 10116 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:49.153868 10116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:49.154124 10116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:49.157050 10116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "950479bed9184eee97bece263ba7088b"
format_stamp: "Formatted at 2026-08-11 12:51:49 on dist-test-slave-f0tn"
server_key: "73521102c409e4fe5f4e4b5f2639b726"
server_key_iv: "74010cfe7911293dc81f8bec5c3e6cd5"
server_key_version: "encryptionkey@0"
I20260811 12:51:49.157822 10116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "950479bed9184eee97bece263ba7088b"
format_stamp: "Formatted at 2026-08-11 12:51:49 on dist-test-slave-f0tn"
server_key: "73521102c409e4fe5f4e4b5f2639b726"
server_key_iv: "74010cfe7911293dc81f8bec5c3e6cd5"
server_key_version: "encryptionkey@0"
I20260811 12:51:49.162833 10116 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 12:51:49.166297 10141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:49.167580 10116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 12:51:49.167742 10116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "950479bed9184eee97bece263ba7088b"
format_stamp: "Formatted at 2026-08-11 12:51:49 on dist-test-slave-f0tn"
server_key: "73521102c409e4fe5f4e4b5f2639b726"
server_key_iv: "74010cfe7911293dc81f8bec5c3e6cd5"
server_key_version: "encryptionkey@0"
I20260811 12:51:49.167929 10116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:49.173542 10116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:49.177290 10116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:49.177618 10116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:49.194458 10116 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41261
I20260811 12:51:49.194500 10237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41261 every 8 connection(s)
I20260811 12:51:49.195925 10116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:51:49.199726 10238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:49.205655 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 10116
I20260811 12:51:49.205825 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:51:49.206254 7613 external_mini_cluster.cc:1428] Setting key 59783b28ee23ced4756461750c139d0c
I20260811 12:51:49.210530 10238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 950479bed9184eee97bece263ba7088b: Bootstrap starting.
I20260811 12:51:49.214007 10245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:49.207987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54763 (local address 127.0.0.1:41261)
0811 12:51:49.208857 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:49.208868 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:49.208910 (+ 42us) server_negotiation.cc:407] Connection header received
0811 12:51:49.210579 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:49.210604 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:49.210964 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:49.211283 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:49.212421 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:49.212427 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:49.212441 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:49.212456 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:49.212525 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:49.212545 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:49.212731 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:49.212831 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 12:51:49.213000 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":400,"threads_started":1}
I20260811 12:51:49.214212 10238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 950479bed9184eee97bece263ba7088b: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:49.215569 10238 log.cc:824] T 00000000000000000000000000000000 P 950479bed9184eee97bece263ba7088b: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:49.218060 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:51:49.218863 10238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 950479bed9184eee97bece263ba7088b: No bootstrap required, opened a new log
I20260811 12:51:49.225843 10245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:49.223283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56373 (local address 127.0.0.1:41261)
0811 12:51:49.223497 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:49.223504 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:49.223900 (+ 396us) server_negotiation.cc:407] Connection header received
0811 12:51:49.224107 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:49.224111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:49.224167 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:49.224250 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:49.225184 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:49.225189 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:49.225195 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:49.225220 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:49.225263 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:49.225279 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:49.225403 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:49.225527 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 12:51:49.225573 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:51:49.228580 10238 raft_consensus.cc:348] T 00000000000000000000000000000000 P 950479bed9184eee97bece263ba7088b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950479bed9184eee97bece263ba7088b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41261 } }
I20260811 12:51:49.228955 10238 raft_consensus.cc:374] T 00000000000000000000000000000000 P 950479bed9184eee97bece263ba7088b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:49.229055 10238 raft_consensus.cc:729] T 00000000000000000000000000000000 P 950479bed9184eee97bece263ba7088b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 950479bed9184eee97bece263ba7088b, State: Initialized, Role: FOLLOWER
I20260811 12:51:49.229666 10238 consensus_queue.cc:260] T 00000000000000000000000000000000 P 950479bed9184eee97bece263ba7088b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950479bed9184eee97bece263ba7088b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41261 } }
I20260811 12:51:49.229903 10238 raft_consensus.cc:388] T 00000000000000000000000000000000 P 950479bed9184eee97bece263ba7088b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:51:49.230034 10238 raft_consensus.cc:482] T 00000000000000000000000000000000 P 950479bed9184eee97bece263ba7088b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:51:49.230197 10238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 950479bed9184eee97bece263ba7088b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:49.232607 10238 raft_consensus.cc:504] T 00000000000000000000000000000000 P 950479bed9184eee97bece263ba7088b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950479bed9184eee97bece263ba7088b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41261 } }
I20260811 12:51:49.233099 10238 leader_election.cc:302] T 00000000000000000000000000000000 P 950479bed9184eee97bece263ba7088b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 950479bed9184eee97bece263ba7088b; no voters:
I20260811 12:51:49.233691 10238 leader_election.cc:288] T 00000000000000000000000000000000 P 950479bed9184eee97bece263ba7088b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:51:49.233824 10255 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 950479bed9184eee97bece263ba7088b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:51:49.234477 10255 raft_consensus.cc:686] T 00000000000000000000000000000000 P 950479bed9184eee97bece263ba7088b [term 1 LEADER]: Becoming Leader. State: Replica: 950479bed9184eee97bece263ba7088b, State: Running, Role: LEADER
I20260811 12:51:49.234977 10255 consensus_queue.cc:237] T 00000000000000000000000000000000 P 950479bed9184eee97bece263ba7088b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950479bed9184eee97bece263ba7088b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41261 } }
I20260811 12:51:49.235688 10238 sys_catalog.cc:564] T 00000000000000000000000000000000 P 950479bed9184eee97bece263ba7088b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:51:49.238884 10262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950479bed9184eee97bece263ba7088b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 950479bed9184eee97bece263ba7088b. Latest consensus state: current_term: 1 leader_uuid: "950479bed9184eee97bece263ba7088b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950479bed9184eee97bece263ba7088b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41261 } } }
I20260811 12:51:49.239223 10261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950479bed9184eee97bece263ba7088b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "950479bed9184eee97bece263ba7088b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950479bed9184eee97bece263ba7088b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41261 } } }
I20260811 12:51:49.239518 10261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950479bed9184eee97bece263ba7088b [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:49.239503 10262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950479bed9184eee97bece263ba7088b [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:49.240455 10277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:51:49.245270 10277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:51:49.252900 10277 catalog_manager.cc:1347] Generated new cluster ID: 35b95dd03b314b918c3f5044832d3b2f
I20260811 12:51:49.253002 10277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:51:49.269155 10277 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:51:49.269687 10277 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:51:49.280849 10277 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 950479bed9184eee97bece263ba7088b: Generated new TSK 0
I20260811 12:51:49.281955 10277 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:51:49.286159 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 10116
W20260811 12:51:49.287168 10229 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54763
W20260811 12:51:49.287828 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 10116: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:51:49.287952 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 10116
2026-08-11T12:51:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2138 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:51:49 dist-test-slave-f0tn krb5kdc[10297](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:51:49 dist-test-slave-f0tn krb5kdc[10297](info): set up 2 sockets
Aug 11 12:51:49 dist-test-slave-f0tn krb5kdc[10297](info): commencing operation
krb5kdc: starting...
W20260811 12:51:50.557497 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.006s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:51:50 dist-test-slave-f0tn krb5kdc[10297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:51:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:51:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:51:50.884285 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45687
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45687
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:51:51.225181 10317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:51.225605 10317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:51.225759 10317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:51.233227 10317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:51:51.233371 10317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:51.233480 10317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:51.233561 10317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:51:51.233621 10317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:51:51.243981 10317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45687
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:51.245368 10317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:51.246757 10317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:51.258435 10332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:51.258481 10327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:51.258458 10329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:51.259547 10317 server_base.cc:1034] running on GCE node
I20260811 12:51:51.260327 10317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:51.261806 10317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:51.263010 10317 hybrid_clock.cc:648] HybridClock initialized: now 1786452711262960 us; error 45 us; skew 500 ppm
I20260811 12:51:51.263368 10317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:51:51 dist-test-slave-f0tn krb5kdc[10297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452711, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:51:51.270538 10317 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:51:51.272141 10317 webserver.cc:460] Webserver started at http://127.0.0.1:42237/ using document root <none> and password file <none>
I20260811 12:51:51.272886 10317 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:51.273010 10317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:51.273313 10317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:51.276477 10317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "912f0b0d7b704eb380871abc18db121b"
format_stamp: "Formatted at 2026-08-11 12:51:51 on dist-test-slave-f0tn"
server_key: "0f22370b1c62061be3e5baa4d01dd130"
server_key_iv: "e4c99b62b9948f37d3ecdfc29b990d27"
server_key_version: "encryptionkey@0"
I20260811 12:51:51.277244 10317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "912f0b0d7b704eb380871abc18db121b"
format_stamp: "Formatted at 2026-08-11 12:51:51 on dist-test-slave-f0tn"
server_key: "0f22370b1c62061be3e5baa4d01dd130"
server_key_iv: "e4c99b62b9948f37d3ecdfc29b990d27"
server_key_version: "encryptionkey@0"
I20260811 12:51:51.282569 10317 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 12:51:51.285974 10345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:51.287228 10317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:51:51.287391 10317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "912f0b0d7b704eb380871abc18db121b"
format_stamp: "Formatted at 2026-08-11 12:51:51 on dist-test-slave-f0tn"
server_key: "0f22370b1c62061be3e5baa4d01dd130"
server_key_iv: "e4c99b62b9948f37d3ecdfc29b990d27"
server_key_version: "encryptionkey@0"
I20260811 12:51:51.287571 10317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:51.301334 10317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:51.305451 10317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:51.305730 10317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:51.323791 10317 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45687
I20260811 12:51:51.323832 10426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45687 every 8 connection(s)
I20260811 12:51:51.325277 10317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:51:51.329053 10428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:51.335635 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 10317
I20260811 12:51:51.335827 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:51:51.336246 7613 external_mini_cluster.cc:1428] Setting key 25081d2136482c31c9cf908efa37fb1a
I20260811 12:51:51.340051 10428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 912f0b0d7b704eb380871abc18db121b: Bootstrap starting.
I20260811 12:51:51.343819 10428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 912f0b0d7b704eb380871abc18db121b: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:51.344869 10428 log.cc:824] T 00000000000000000000000000000000 P 912f0b0d7b704eb380871abc18db121b: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:51.347990 10428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 912f0b0d7b704eb380871abc18db121b: No bootstrap required, opened a new log
Aug 11 12:51:51 dist-test-slave-f0tn krb5kdc[10297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 12:51:51.357407 10428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 912f0b0d7b704eb380871abc18db121b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912f0b0d7b704eb380871abc18db121b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45687 } }
I20260811 12:51:51.357842 10428 raft_consensus.cc:374] T 00000000000000000000000000000000 P 912f0b0d7b704eb380871abc18db121b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:51.357957 10428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 912f0b0d7b704eb380871abc18db121b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 912f0b0d7b704eb380871abc18db121b, State: Initialized, Role: FOLLOWER
I20260811 12:51:51.358695 10428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 912f0b0d7b704eb380871abc18db121b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912f0b0d7b704eb380871abc18db121b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45687 } }
I20260811 12:51:51.358942 10428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 912f0b0d7b704eb380871abc18db121b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:51:51.359081 10428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 912f0b0d7b704eb380871abc18db121b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:51:51.359246 10428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 912f0b0d7b704eb380871abc18db121b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:51.360020 10433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:51.338149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51473 (local address 127.0.0.1:45687)
0811 12:51:51.338954 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:51.338967 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:51.339012 (+ 45us) server_negotiation.cc:407] Connection header received
0811 12:51:51.340783 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:51.340808 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:51.341220 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:51.341603 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:51.343481 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:51.344833 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:51.346378 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:51.347036 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:51.347163 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:51.352217 (+ 5054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:51.352226 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:51.352245 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:51.352292 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:51.355498 (+ 3206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:51.356376 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:51.356384 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:51.356391 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:51.356576 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:51.356978 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:51.356981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:51.356983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:51.357606 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:51.357878 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:51.358365 (+ 487us) server_negotiation.cc:299] Negotiation successful
0811 12:51:51.358796 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":336,"threads_started":1}
I20260811 12:51:51.361788 10428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 912f0b0d7b704eb380871abc18db121b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912f0b0d7b704eb380871abc18db121b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45687 } }
I20260811 12:51:51.362285 10428 leader_election.cc:302] T 00000000000000000000000000000000 P 912f0b0d7b704eb380871abc18db121b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 912f0b0d7b704eb380871abc18db121b; no voters:
I20260811 12:51:51.362881 10428 leader_election.cc:288] T 00000000000000000000000000000000 P 912f0b0d7b704eb380871abc18db121b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:51:51.363032 10436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 912f0b0d7b704eb380871abc18db121b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:51:51.364064 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:51:51.364207 10436 raft_consensus.cc:686] T 00000000000000000000000000000000 P 912f0b0d7b704eb380871abc18db121b [term 1 LEADER]: Becoming Leader. State: Replica: 912f0b0d7b704eb380871abc18db121b, State: Running, Role: LEADER
I20260811 12:51:51.364743 10436 consensus_queue.cc:237] T 00000000000000000000000000000000 P 912f0b0d7b704eb380871abc18db121b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912f0b0d7b704eb380871abc18db121b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45687 } }
I20260811 12:51:51.365012 10428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 912f0b0d7b704eb380871abc18db121b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:51:51.371774 10438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 912f0b0d7b704eb380871abc18db121b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 912f0b0d7b704eb380871abc18db121b. Latest consensus state: current_term: 1 leader_uuid: "912f0b0d7b704eb380871abc18db121b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912f0b0d7b704eb380871abc18db121b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45687 } } }
I20260811 12:51:51.372227 10438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 912f0b0d7b704eb380871abc18db121b [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:51.372819 10437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 912f0b0d7b704eb380871abc18db121b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "912f0b0d7b704eb380871abc18db121b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912f0b0d7b704eb380871abc18db121b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45687 } } }
I20260811 12:51:51.373102 10437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 912f0b0d7b704eb380871abc18db121b [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:51.376785 10457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:51:51.382995 10457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:51:51.398270 10433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:51.378551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56681 (local address 127.0.0.1:45687)
0811 12:51:51.378747 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:51.378753 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:51.379272 (+ 519us) server_negotiation.cc:407] Connection header received
0811 12:51:51.379545 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:51.379551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:51.379629 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:51.379723 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:51.381246 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:51.382419 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:51.384062 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:51.384834 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:51.384945 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:51.390773 (+ 5828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:51.390785 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:51.390790 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:51.390832 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:51.394748 (+ 3916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:51.395657 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:51.395662 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:51.395666 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:51.395832 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:51.396301 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:51.396306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:51.396310 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:51.396745 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:51.396975 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:51.397642 (+ 667us) server_negotiation.cc:299] Negotiation successful
0811 12:51:51.397878 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:51:51.401872 10457 catalog_manager.cc:1347] Generated new cluster ID: 997626bc23b14379b84308d3d4b30fbc
I20260811 12:51:51.401995 10457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:51:51.424976 10457 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:51:51.426963 10457 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:51:51.433303 10457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 912f0b0d7b704eb380871abc18db121b: Generated new TSK 0
I20260811 12:51:51.434130 10457 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:51:51.447602 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 10317
I20260811 12:51:51.448693 10419 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:51.542642 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 10317
2026-08-11T12:51:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2250 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:51:51 dist-test-slave-f0tn krb5kdc[10480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:51:51 dist-test-slave-f0tn krb5kdc[10480](info): set up 2 sockets
Aug 11 12:51:51 dist-test-slave-f0tn krb5kdc[10480](info): commencing operation
krb5kdc: starting...
W20260811 12:51:52.823722 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.002s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:51:53 dist-test-slave-f0tn krb5kdc[10480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:51:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:51:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:51:53.150239 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38029
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:51:53.494989 10504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:53.495270 10504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:53.495361 10504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:53.502442 10504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:51:53.502552 10504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:53.502597 10504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:53.502656 10504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:51:53.502699 10504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:51:53.512379 10504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:53.513593 10504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:53.514849 10504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:53.526072 10521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:53.526079 10528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:53.526371 10520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:53.526871 10504 server_base.cc:1034] running on GCE node
I20260811 12:51:53.527666 10504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:53.529047 10504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:53.530238 10504 hybrid_clock.cc:648] HybridClock initialized: now 1786452713530187 us; error 44 us; skew 500 ppm
I20260811 12:51:53.530572 10504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:51:53 dist-test-slave-f0tn krb5kdc[10480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452713, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:51:53.533560 10504 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:51:53.534927 10504 webserver.cc:460] Webserver started at http://127.0.0.1:32861/ using document root <none> and password file <none>
I20260811 12:51:53.535594 10504 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:53.535710 10504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:53.536007 10504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:53.539037 10504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "6b87760dc6f04c3fa5cc457f30b171e6"
format_stamp: "Formatted at 2026-08-11 12:51:53 on dist-test-slave-f0tn"
server_key: "09e5d6a36fa3dac83cfda2792524050a"
server_key_iv: "3c99f249f458055230ae20eb6f4ff7f3"
server_key_version: "encryptionkey@0"
I20260811 12:51:53.539769 10504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "6b87760dc6f04c3fa5cc457f30b171e6"
format_stamp: "Formatted at 2026-08-11 12:51:53 on dist-test-slave-f0tn"
server_key: "09e5d6a36fa3dac83cfda2792524050a"
server_key_iv: "3c99f249f458055230ae20eb6f4ff7f3"
server_key_version: "encryptionkey@0"
I20260811 12:51:53.544803 10504 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 12:51:53.548190 10542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:53.549467 10504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:51:53.549626 10504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "6b87760dc6f04c3fa5cc457f30b171e6"
format_stamp: "Formatted at 2026-08-11 12:51:53 on dist-test-slave-f0tn"
server_key: "09e5d6a36fa3dac83cfda2792524050a"
server_key_iv: "3c99f249f458055230ae20eb6f4ff7f3"
server_key_version: "encryptionkey@0"
I20260811 12:51:53.549805 10504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:53.562613 10504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:53.566589 10504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:53.566869 10504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:53.585472 10504 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38029
I20260811 12:51:53.585465 10628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38029 every 8 connection(s)
I20260811 12:51:53.587034 10504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:51:53.590791 10629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:53.593685 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 10504
I20260811 12:51:53.593880 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:51:53.594416 7613 external_mini_cluster.cc:1428] Setting key 23cffc894589f0e216d788530f0e2f20
I20260811 12:51:53.603210 10629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b87760dc6f04c3fa5cc457f30b171e6: Bootstrap starting.
I20260811 12:51:53.606809 10629 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b87760dc6f04c3fa5cc457f30b171e6: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:53.607951 10629 log.cc:824] T 00000000000000000000000000000000 P 6b87760dc6f04c3fa5cc457f30b171e6: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:53.608374 10634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:53.596270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59611 (local address 127.0.0.1:38029)
0811 12:51:53.597014 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:53.597026 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:53.597067 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:51:53.599119 (+ 2052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:53.599142 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:53.599541 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:53.599861 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:53.600444 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:53.602114 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:53.604121 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:53.604829 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:53.604966 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:53.606167 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:53.606175 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:53.606189 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:53.606709 (+ 520us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:53.606782 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:53.606810 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:53.606908 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:53.607120 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 12:51:53.607373 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":332,"threads_started":1}
I20260811 12:51:53.610855 10629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b87760dc6f04c3fa5cc457f30b171e6: No bootstrap required, opened a new log
I20260811 12:51:53.612037 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:51:53.620554 10629 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b87760dc6f04c3fa5cc457f30b171e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b87760dc6f04c3fa5cc457f30b171e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38029 } }
I20260811 12:51:53.620994 10629 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b87760dc6f04c3fa5cc457f30b171e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:53.621114 10629 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b87760dc6f04c3fa5cc457f30b171e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b87760dc6f04c3fa5cc457f30b171e6, State: Initialized, Role: FOLLOWER
I20260811 12:51:53.621793 10629 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b87760dc6f04c3fa5cc457f30b171e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b87760dc6f04c3fa5cc457f30b171e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38029 } }
I20260811 12:51:53.622087 10629 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b87760dc6f04c3fa5cc457f30b171e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:51:53.622244 10629 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b87760dc6f04c3fa5cc457f30b171e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:51:53.622421 10629 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b87760dc6f04c3fa5cc457f30b171e6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:53.623488 10634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:53.617141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53529 (local address 127.0.0.1:38029)
0811 12:51:53.617299 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:53.617304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:53.617630 (+ 326us) server_negotiation.cc:407] Connection header received
0811 12:51:53.617812 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:53.617816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:53.617876 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:53.617949 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:53.618510 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:53.619286 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:53.621087 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:53.621657 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:53.621764 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:53.622880 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:53.622884 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:53.622887 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:53.622904 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:53.622943 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:53.622958 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:53.623030 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:53.623163 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 12:51:53.623233 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 12:51:53.625259 10629 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b87760dc6f04c3fa5cc457f30b171e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b87760dc6f04c3fa5cc457f30b171e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38029 } }
I20260811 12:51:53.625773 10629 leader_election.cc:302] T 00000000000000000000000000000000 P 6b87760dc6f04c3fa5cc457f30b171e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b87760dc6f04c3fa5cc457f30b171e6; no voters:
I20260811 12:51:53.626335 10629 leader_election.cc:288] T 00000000000000000000000000000000 P 6b87760dc6f04c3fa5cc457f30b171e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:51:53.626448 10648 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b87760dc6f04c3fa5cc457f30b171e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:51:53.627092 10648 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b87760dc6f04c3fa5cc457f30b171e6 [term 1 LEADER]: Becoming Leader. State: Replica: 6b87760dc6f04c3fa5cc457f30b171e6, State: Running, Role: LEADER
I20260811 12:51:53.627794 10648 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b87760dc6f04c3fa5cc457f30b171e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b87760dc6f04c3fa5cc457f30b171e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38029 } }
I20260811 12:51:53.628472 10629 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b87760dc6f04c3fa5cc457f30b171e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:51:53.630666 10653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b87760dc6f04c3fa5cc457f30b171e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b87760dc6f04c3fa5cc457f30b171e6. Latest consensus state: current_term: 1 leader_uuid: "6b87760dc6f04c3fa5cc457f30b171e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b87760dc6f04c3fa5cc457f30b171e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38029 } } }
I20260811 12:51:53.631067 10653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b87760dc6f04c3fa5cc457f30b171e6 [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:53.631448 10652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b87760dc6f04c3fa5cc457f30b171e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b87760dc6f04c3fa5cc457f30b171e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b87760dc6f04c3fa5cc457f30b171e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38029 } } }
I20260811 12:51:53.631747 10652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b87760dc6f04c3fa5cc457f30b171e6 [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:53.633833 10662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:51:53.638972 10662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:51:53.647751 10662 catalog_manager.cc:1347] Generated new cluster ID: bbde55705d00478b99ab36a3bef2654a
I20260811 12:51:53.647863 10662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:51:53.680869 10662 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:51:53.682276 10662 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:51:53.693588 10662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b87760dc6f04c3fa5cc457f30b171e6: Generated new TSK 0
I20260811 12:51:53.694412 10662 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:51:53.702644 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 10504
W20260811 12:51:53.703727 10622 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59611
W20260811 12:51:53.704530 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 10504: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:51:53.704702 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 10504
2026-08-11T12:51:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2162 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:51:53 dist-test-slave-f0tn krb5kdc[10682](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:51:53 dist-test-slave-f0tn krb5kdc[10682](info): set up 2 sockets
Aug 11 12:51:53 dist-test-slave-f0tn krb5kdc[10682](info): commencing operation
krb5kdc: starting...
W20260811 12:51:54.980563 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:51:55 dist-test-slave-f0tn krb5kdc[10682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:51:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:51:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:51:55.330658 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44195
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:51:55.676817 10704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:55.677106 10704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:55.677201 10704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:55.684304 10704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:51:55.684412 10704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:55.684466 10704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:55.684525 10704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:51:55.684568 10704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:51:55.694128 10704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:55.695374 10704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:55.696656 10704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:55.708425 10712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:55.708369 10714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:55.708418 10721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:55.709236 10704 server_base.cc:1034] running on GCE node
I20260811 12:51:55.710081 10704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:55.711506 10704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:55.712738 10704 hybrid_clock.cc:648] HybridClock initialized: now 1786452715712674 us; error 46 us; skew 500 ppm
I20260811 12:51:55.713109 10704 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:51:55 dist-test-slave-f0tn krb5kdc[10682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452715, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:51:55.716254 10704 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:51:55.717821 10704 webserver.cc:460] Webserver started at http://127.0.0.1:42571/ using document root <none> and password file <none>
I20260811 12:51:55.718538 10704 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:55.718673 10704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:55.719026 10704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:55.722294 10704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "8cc36f4af7654e349517fffa4ee2610f"
format_stamp: "Formatted at 2026-08-11 12:51:55 on dist-test-slave-f0tn"
server_key: "5b9257ca33cfaa405de68aa1f5b5e690"
server_key_iv: "0b6637b5b70e906c1ef5554f7ab8ae18"
server_key_version: "encryptionkey@0"
I20260811 12:51:55.723126 10704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "8cc36f4af7654e349517fffa4ee2610f"
format_stamp: "Formatted at 2026-08-11 12:51:55 on dist-test-slave-f0tn"
server_key: "5b9257ca33cfaa405de68aa1f5b5e690"
server_key_iv: "0b6637b5b70e906c1ef5554f7ab8ae18"
server_key_version: "encryptionkey@0"
I20260811 12:51:55.728744 10704 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:51:55.732448 10731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:55.733861 10704 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:51:55.734040 10704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "8cc36f4af7654e349517fffa4ee2610f"
format_stamp: "Formatted at 2026-08-11 12:51:55 on dist-test-slave-f0tn"
server_key: "5b9257ca33cfaa405de68aa1f5b5e690"
server_key_iv: "0b6637b5b70e906c1ef5554f7ab8ae18"
server_key_version: "encryptionkey@0"
I20260811 12:51:55.734248 10704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:55.740235 10704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:55.744172 10704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:55.744460 10704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:55.762596 10704 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44195
I20260811 12:51:55.762687 10810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44195 every 8 connection(s)
I20260811 12:51:55.764084 10704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:51:55.764582 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 10704
I20260811 12:51:55.764765 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:51:55.765319 7613 external_mini_cluster.cc:1428] Setting key 71b87de019e5806a77cca08bdf9fccba
I20260811 12:51:55.768599 10812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:55.773044 10815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:55.767164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55667 (local address 127.0.0.1:44195)
0811 12:51:55.767946 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:55.767959 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:55.768000 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:51:55.769696 (+ 1696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:55.769722 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:55.770049 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:55.770411 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:55.771449 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:55.771456 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:55.771473 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:55.771488 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:55.771569 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:55.771593 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:55.771805 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:55.771939 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 12:51:55.772133 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":317,"threads_started":1}
I20260811 12:51:55.777148 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:51:55.780376 10812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cc36f4af7654e349517fffa4ee2610f: Bootstrap starting.
I20260811 12:51:55.783898 10815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:55.781415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38347 (local address 127.0.0.1:44195)
0811 12:51:55.781587 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:55.781593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:55.782000 (+ 407us) server_negotiation.cc:407] Connection header received
0811 12:51:55.782139 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:55.782144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:55.782218 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:55.782330 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:55.783277 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:55.783283 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:55.783287 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:51:55.783306 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:55.783357 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:51:55.783377 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:55.783482 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:55.783571 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 12:51:55.783630 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:51:55.784101 10812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cc36f4af7654e349517fffa4ee2610f: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:55.785431 10812 log.cc:824] T 00000000000000000000000000000000 P 8cc36f4af7654e349517fffa4ee2610f: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:55.788244 10812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cc36f4af7654e349517fffa4ee2610f: No bootstrap required, opened a new log
I20260811 12:51:55.795951 10812 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cc36f4af7654e349517fffa4ee2610f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc36f4af7654e349517fffa4ee2610f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44195 } }
I20260811 12:51:55.796352 10812 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cc36f4af7654e349517fffa4ee2610f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:55.796460 10812 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cc36f4af7654e349517fffa4ee2610f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cc36f4af7654e349517fffa4ee2610f, State: Initialized, Role: FOLLOWER
I20260811 12:51:55.797106 10812 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cc36f4af7654e349517fffa4ee2610f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc36f4af7654e349517fffa4ee2610f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44195 } }
I20260811 12:51:55.797317 10812 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cc36f4af7654e349517fffa4ee2610f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:51:55.797456 10812 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cc36f4af7654e349517fffa4ee2610f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:51:55.797602 10812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cc36f4af7654e349517fffa4ee2610f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:55.799742 10812 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cc36f4af7654e349517fffa4ee2610f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc36f4af7654e349517fffa4ee2610f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44195 } }
I20260811 12:51:55.800194 10812 leader_election.cc:302] T 00000000000000000000000000000000 P 8cc36f4af7654e349517fffa4ee2610f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cc36f4af7654e349517fffa4ee2610f; no voters:
I20260811 12:51:55.800758 10812 leader_election.cc:288] T 00000000000000000000000000000000 P 8cc36f4af7654e349517fffa4ee2610f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:51:55.800910 10831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cc36f4af7654e349517fffa4ee2610f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:51:55.801736 10831 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cc36f4af7654e349517fffa4ee2610f [term 1 LEADER]: Becoming Leader. State: Replica: 8cc36f4af7654e349517fffa4ee2610f, State: Running, Role: LEADER
I20260811 12:51:55.802364 10831 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cc36f4af7654e349517fffa4ee2610f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc36f4af7654e349517fffa4ee2610f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44195 } }
I20260811 12:51:55.802973 10812 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cc36f4af7654e349517fffa4ee2610f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:51:55.806139 10833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cc36f4af7654e349517fffa4ee2610f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cc36f4af7654e349517fffa4ee2610f. Latest consensus state: current_term: 1 leader_uuid: "8cc36f4af7654e349517fffa4ee2610f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc36f4af7654e349517fffa4ee2610f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44195 } } }
I20260811 12:51:55.806113 10832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cc36f4af7654e349517fffa4ee2610f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cc36f4af7654e349517fffa4ee2610f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc36f4af7654e349517fffa4ee2610f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44195 } } }
I20260811 12:51:55.806702 10833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cc36f4af7654e349517fffa4ee2610f [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:55.806742 10832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cc36f4af7654e349517fffa4ee2610f [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:55.808532 10844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:51:55.814118 10844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:51:55.823607 10844 catalog_manager.cc:1347] Generated new cluster ID: 12d9dbc6b25942e383ed541068ca0bbd
I20260811 12:51:55.823740 10844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:51:55.875888 10844 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:51:55.876327 10844 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:51:55.885921 10844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cc36f4af7654e349517fffa4ee2610f: Generated new TSK 0
I20260811 12:51:55.886766 10844 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:51:55.900786 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 10704
W20260811 12:51:55.902454 10804 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55667
W20260811 12:51:55.903241 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 10704: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:51:55.903388 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 10704
2026-08-11T12:51:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2200 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:51:56 dist-test-slave-f0tn krb5kdc[10860](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:51:56 dist-test-slave-f0tn krb5kdc[10860](info): set up 2 sockets
Aug 11 12:51:56 dist-test-slave-f0tn krb5kdc[10860](info): commencing operation
krb5kdc: starting...
W20260811 12:51:57.190488 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.000s sys 0.058s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:51:57 dist-test-slave-f0tn krb5kdc[10860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:51:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:51:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:51:57.546481 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37121
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:51:57.892930 10882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:51:57.893258 10882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:51:57.893435 10882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:51:57.901100 10882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:51:57.901230 10882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:51:57.901316 10882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:51:57.901437 10882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:51:57.901503 10882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:51:57.911928 10882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:51:57.913326 10882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:51:57.914776 10882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:51:57.926904 10893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:57.926939 10894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:51:57.926862 10899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:51:57.928054 10882 server_base.cc:1034] running on GCE node
I20260811 12:51:57.928867 10882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:51:57.930370 10882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:51:57.931608 10882 hybrid_clock.cc:648] HybridClock initialized: now 1786452717931547 us; error 49 us; skew 500 ppm
I20260811 12:51:57.931977 10882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:51:57 dist-test-slave-f0tn krb5kdc[10860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452717, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:51:57.938726 10882 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:51:57.940387 10882 webserver.cc:460] Webserver started at http://127.0.0.1:35333/ using document root <none> and password file <none>
I20260811 12:51:57.941109 10882 fs_manager.cc:359] Metadata directory not provided
I20260811 12:51:57.941236 10882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:51:57.941592 10882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:51:57.944674 10882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "9d285d58fadb46a4975cbf5c89b621fd"
format_stamp: "Formatted at 2026-08-11 12:51:57 on dist-test-slave-f0tn"
server_key: "0fdf0b9cbfd5642e28afec36c9a9014b"
server_key_iv: "2b5c57c24b066bb37dd54e2288d6600b"
server_key_version: "encryptionkey@0"
I20260811 12:51:57.945478 10882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "9d285d58fadb46a4975cbf5c89b621fd"
format_stamp: "Formatted at 2026-08-11 12:51:57 on dist-test-slave-f0tn"
server_key: "0fdf0b9cbfd5642e28afec36c9a9014b"
server_key_iv: "2b5c57c24b066bb37dd54e2288d6600b"
server_key_version: "encryptionkey@0"
I20260811 12:51:57.950654 10882 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 12:51:57.954303 10911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:51:57.955576 10882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:51:57.955736 10882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "9d285d58fadb46a4975cbf5c89b621fd"
format_stamp: "Formatted at 2026-08-11 12:51:57 on dist-test-slave-f0tn"
server_key: "0fdf0b9cbfd5642e28afec36c9a9014b"
server_key_iv: "2b5c57c24b066bb37dd54e2288d6600b"
server_key_version: "encryptionkey@0"
I20260811 12:51:57.955914 10882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:51:57.967470 10882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:51:57.971479 10882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:51:57.971736 10882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:51:57.988847 10882 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37121
I20260811 12:51:57.988873 11008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37121 every 8 connection(s)
I20260811 12:51:57.990326 10882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:51:57.994046 11009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:51:58.000200 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 10882
I20260811 12:51:58.000419 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:51:58.000958 7613 external_mini_cluster.cc:1428] Setting key 25f521b695ff4e040285c61ce3832b61
I20260811 12:51:58.004899 11009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d285d58fadb46a4975cbf5c89b621fd: Bootstrap starting.
I20260811 12:51:58.008559 11009 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d285d58fadb46a4975cbf5c89b621fd: Neither blocks nor log segments found. Creating new log.
I20260811 12:51:58.009681 11009 log.cc:824] T 00000000000000000000000000000000 P 9d285d58fadb46a4975cbf5c89b621fd: Log is configured to *not* fsync() on all Append() calls
I20260811 12:51:58.012806 11009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d285d58fadb46a4975cbf5c89b621fd: No bootstrap required, opened a new log
Aug 11 12:51:58 dist-test-slave-f0tn krb5kdc[10860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 12:51:58.022105 11009 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d285d58fadb46a4975cbf5c89b621fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d285d58fadb46a4975cbf5c89b621fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37121 } }
I20260811 12:51:58.022505 11009 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d285d58fadb46a4975cbf5c89b621fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:51:58.022629 11009 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d285d58fadb46a4975cbf5c89b621fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d285d58fadb46a4975cbf5c89b621fd, State: Initialized, Role: FOLLOWER
I20260811 12:51:58.023429 11009 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d285d58fadb46a4975cbf5c89b621fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d285d58fadb46a4975cbf5c89b621fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37121 } }
I20260811 12:51:58.023697 11009 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d285d58fadb46a4975cbf5c89b621fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:51:58.023856 11009 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d285d58fadb46a4975cbf5c89b621fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:51:58.024040 11009 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d285d58fadb46a4975cbf5c89b621fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:51:58.024662 11013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:58.003018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:56801 (local address 127.0.0.1:37121)
0811 12:51:58.003804 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:58.003816 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:58.003858 (+ 42us) server_negotiation.cc:407] Connection header received
0811 12:51:58.005569 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:58.005593 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:58.005984 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:58.006311 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:58.008065 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:58.009462 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:58.011001 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:58.011652 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:58.011797 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:58.016817 (+ 5020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:58.016828 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:58.016845 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:58.016886 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:58.020103 (+ 3217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:58.021065 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:58.021072 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:58.021079 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:58.021262 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:58.021764 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:58.021767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:58.021769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:58.022305 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:58.022593 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:58.023111 (+ 518us) server_negotiation.cc:299] Negotiation successful
0811 12:51:58.023442 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":351,"threads_started":1}
I20260811 12:51:58.026542 11009 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d285d58fadb46a4975cbf5c89b621fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d285d58fadb46a4975cbf5c89b621fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37121 } }
I20260811 12:51:58.027050 11009 leader_election.cc:302] T 00000000000000000000000000000000 P 9d285d58fadb46a4975cbf5c89b621fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d285d58fadb46a4975cbf5c89b621fd; no voters:
I20260811 12:51:58.027683 11009 leader_election.cc:288] T 00000000000000000000000000000000 P 9d285d58fadb46a4975cbf5c89b621fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:51:58.027792 11018 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d285d58fadb46a4975cbf5c89b621fd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:51:58.028726 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:51:58.029040 11018 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d285d58fadb46a4975cbf5c89b621fd [term 1 LEADER]: Becoming Leader. State: Replica: 9d285d58fadb46a4975cbf5c89b621fd, State: Running, Role: LEADER
I20260811 12:51:58.029559 11018 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d285d58fadb46a4975cbf5c89b621fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d285d58fadb46a4975cbf5c89b621fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37121 } }
I20260811 12:51:58.029891 11009 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d285d58fadb46a4975cbf5c89b621fd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:51:58.032195 11021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d285d58fadb46a4975cbf5c89b621fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d285d58fadb46a4975cbf5c89b621fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d285d58fadb46a4975cbf5c89b621fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37121 } } }
I20260811 12:51:58.032477 11021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d285d58fadb46a4975cbf5c89b621fd [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:58.032102 11024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d285d58fadb46a4975cbf5c89b621fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d285d58fadb46a4975cbf5c89b621fd. Latest consensus state: current_term: 1 leader_uuid: "9d285d58fadb46a4975cbf5c89b621fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d285d58fadb46a4975cbf5c89b621fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37121 } } }
I20260811 12:51:58.032955 11024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d285d58fadb46a4975cbf5c89b621fd [sys.catalog]: This master's current role is: LEADER
I20260811 12:51:58.046283 11030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:51:58.051393 11030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:51:58.064857 11030 catalog_manager.cc:1347] Generated new cluster ID: 80d4ef5883704d5986ba15df8e8fd6ef
I20260811 12:51:58.064986 11030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:51:58.085526 11013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:51:58.053130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:41495 (local address 127.0.0.1:37121)
0811 12:51:58.053332 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 12:51:58.053340 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:51:58.066497 (+ 13157us) server_negotiation.cc:407] Connection header received
0811 12:51:58.067036 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:51:58.067043 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:51:58.067135 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:51:58.067222 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:51:58.069557 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:58.070853 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:58.072830 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:51:58.073563 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:51:58.073666 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:51:58.077292 (+ 3626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:51:58.077300 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:51:58.077305 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:51:58.077350 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:51:58.081424 (+ 4074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:58.082604 (+ 1180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:58.082610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:58.082614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:58.082814 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:51:58.083443 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:51:58.083450 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:51:58.083454 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:51:58.083856 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:51:58.084106 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 12:51:58.084846 (+ 740us) server_negotiation.cc:299] Negotiation successful
0811 12:51:58.085049 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:51:58.100436 11030 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:51:58.101960 11030 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:51:58.108989 11030 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d285d58fadb46a4975cbf5c89b621fd: Generated new TSK 0
I20260811 12:51:58.109884 11030 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:51:58.115828 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 10882
I20260811 12:51:58.117208 10999 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:51:58.208603 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 10882
W20260811 12:51:58.233876 10858 connection.cc:535] client connection to 127.0.0.1:37121 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37121): Connection reset by peer (error 104)
2026-08-11T12:51:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2301 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:51:58 dist-test-slave-f0tn krb5kdc[11068](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:51:58 dist-test-slave-f0tn krb5kdc[11068](info): set up 2 sockets
Aug 11 12:51:58 dist-test-slave-f0tn krb5kdc[11068](info): commencing operation
krb5kdc: starting...
W20260811 12:51:59.482789 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.000s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:51:59 dist-test-slave-f0tn krb5kdc[11068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:51:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:51:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:51:59.800048 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40459
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:52:00.141494 11090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:52:00.141853 11090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:52:00.141997 11090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:52:00.149501 11090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:52:00.149628 11090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:52:00.149713 11090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:52:00.149801 11090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:52:00.149863 11090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:52:00.160104 11090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:52:00.161479 11090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:52:00.162906 11090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:52:00.177955 11102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:00.177938 11107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:00.178143 11100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:52:00.178876 11090 server_base.cc:1034] running on GCE node
I20260811 12:52:00.179747 11090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:52:00.181252 11090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:52:00.182479 11090 hybrid_clock.cc:648] HybridClock initialized: now 1786452720182438 us; error 48 us; skew 500 ppm
I20260811 12:52:00.182847 11090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:52:00 dist-test-slave-f0tn krb5kdc[11068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452720, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:52:00.186107 11090 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:52:00.187672 11090 webserver.cc:460] Webserver started at http://127.0.0.1:41329/ using document root <none> and password file <none>
I20260811 12:52:00.188410 11090 fs_manager.cc:359] Metadata directory not provided
I20260811 12:52:00.188531 11090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:52:00.188853 11090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:52:00.192055 11090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "11ed54495c1343a09f2054782e9a49bc"
format_stamp: "Formatted at 2026-08-11 12:52:00 on dist-test-slave-f0tn"
server_key: "c0cbb46b7b82a9aa087e0824f663ec51"
server_key_iv: "c16ac9c37cedce4b6dc03a949c4f0695"
server_key_version: "encryptionkey@0"
I20260811 12:52:00.192832 11090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "11ed54495c1343a09f2054782e9a49bc"
format_stamp: "Formatted at 2026-08-11 12:52:00 on dist-test-slave-f0tn"
server_key: "c0cbb46b7b82a9aa087e0824f663ec51"
server_key_iv: "c16ac9c37cedce4b6dc03a949c4f0695"
server_key_version: "encryptionkey@0"
I20260811 12:52:00.198310 11090 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 12:52:00.201877 11118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:52:00.203234 11090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:52:00.203395 11090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "11ed54495c1343a09f2054782e9a49bc"
format_stamp: "Formatted at 2026-08-11 12:52:00 on dist-test-slave-f0tn"
server_key: "c0cbb46b7b82a9aa087e0824f663ec51"
server_key_iv: "c16ac9c37cedce4b6dc03a949c4f0695"
server_key_version: "encryptionkey@0"
I20260811 12:52:00.203580 11090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:52:00.227555 11090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:52:00.232072 11090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:52:00.232362 11090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:52:00.250739 11090 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40459
I20260811 12:52:00.250773 11204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40459 every 8 connection(s)
I20260811 12:52:00.252170 11090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:52:00.256363 11209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:52:00.261850 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 11090
I20260811 12:52:00.262068 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:52:00.262673 7613 external_mini_cluster.cc:1428] Setting key eae19e4151a883802254220edc49c67b
I20260811 12:52:00.269680 11209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11ed54495c1343a09f2054782e9a49bc: Bootstrap starting.
I20260811 12:52:00.274924 11209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11ed54495c1343a09f2054782e9a49bc: Neither blocks nor log segments found. Creating new log.
I20260811 12:52:00.276609 11209 log.cc:824] T 00000000000000000000000000000000 P 11ed54495c1343a09f2054782e9a49bc: Log is configured to *not* fsync() on all Append() calls
I20260811 12:52:00.279826 11216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:00.264784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:49537 (local address 127.0.0.1:40459)
0811 12:52:00.266824 (+ 2040us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:00.266845 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:00.266898 (+ 53us) server_negotiation.cc:407] Connection header received
0811 12:52:00.269305 (+ 2407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:00.269339 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:00.269913 (+ 574us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:00.270371 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:00.270908 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:00.272592 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:00.274228 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:00.275114 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:00.275276 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:00.276929 (+ 1653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:00.276941 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:00.276962 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:52:00.277802 (+ 840us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:00.277905 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:52:00.277937 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:00.278126 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:00.278648 (+ 522us) server_negotiation.cc:299] Negotiation successful
0811 12:52:00.278821 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1754,"thread_start_us":2003,"threads_started":1}
I20260811 12:52:00.280155 11209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11ed54495c1343a09f2054782e9a49bc: No bootstrap required, opened a new log
I20260811 12:52:00.284106 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:52:00.288492 11209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11ed54495c1343a09f2054782e9a49bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11ed54495c1343a09f2054782e9a49bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40459 } }
I20260811 12:52:00.289016 11209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11ed54495c1343a09f2054782e9a49bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:52:00.289124 11209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11ed54495c1343a09f2054782e9a49bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11ed54495c1343a09f2054782e9a49bc, State: Initialized, Role: FOLLOWER
I20260811 12:52:00.289767 11209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11ed54495c1343a09f2054782e9a49bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11ed54495c1343a09f2054782e9a49bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40459 } }
I20260811 12:52:00.289981 11209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11ed54495c1343a09f2054782e9a49bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:52:00.290112 11209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11ed54495c1343a09f2054782e9a49bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:52:00.290280 11209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11ed54495c1343a09f2054782e9a49bc [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:52:00.292521 11209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11ed54495c1343a09f2054782e9a49bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11ed54495c1343a09f2054782e9a49bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40459 } }
I20260811 12:52:00.293030 11209 leader_election.cc:302] T 00000000000000000000000000000000 P 11ed54495c1343a09f2054782e9a49bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11ed54495c1343a09f2054782e9a49bc; no voters:
I20260811 12:52:00.293669 11209 leader_election.cc:288] T 00000000000000000000000000000000 P 11ed54495c1343a09f2054782e9a49bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:52:00.293787 11228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11ed54495c1343a09f2054782e9a49bc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:52:00.294538 11228 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11ed54495c1343a09f2054782e9a49bc [term 1 LEADER]: Becoming Leader. State: Replica: 11ed54495c1343a09f2054782e9a49bc, State: Running, Role: LEADER
I20260811 12:52:00.295138 11228 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11ed54495c1343a09f2054782e9a49bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11ed54495c1343a09f2054782e9a49bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40459 } }
I20260811 12:52:00.295917 11216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:00.288685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:44683 (local address 127.0.0.1:40459)
0811 12:52:00.288859 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:00.288866 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:00.289351 (+ 485us) server_negotiation.cc:407] Connection header received
0811 12:52:00.289595 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:00.289600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:00.289667 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:00.289764 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:00.290406 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:00.291220 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:00.292634 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:00.293193 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:00.293339 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:00.294952 (+ 1613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:00.294959 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:00.294963 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:52:00.294991 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:00.295052 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:52:00.295079 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:00.295224 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:00.295570 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 12:52:00.295627 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 12:52:00.295996 11209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11ed54495c1343a09f2054782e9a49bc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:52:00.298545 11229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11ed54495c1343a09f2054782e9a49bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11ed54495c1343a09f2054782e9a49bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11ed54495c1343a09f2054782e9a49bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40459 } } }
I20260811 12:52:00.298955 11229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11ed54495c1343a09f2054782e9a49bc [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:00.299391 11231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11ed54495c1343a09f2054782e9a49bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11ed54495c1343a09f2054782e9a49bc. Latest consensus state: current_term: 1 leader_uuid: "11ed54495c1343a09f2054782e9a49bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11ed54495c1343a09f2054782e9a49bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40459 } } }
I20260811 12:52:00.299672 11231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11ed54495c1343a09f2054782e9a49bc [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:00.299891 11241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:52:00.308285 11241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:52:00.318439 11241 catalog_manager.cc:1347] Generated new cluster ID: 9f0ac040b8714541b6a88c6f43fac4bd
I20260811 12:52:00.318583 11241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:52:00.332062 11241 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:52:00.333405 11241 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:52:00.340667 11241 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11ed54495c1343a09f2054782e9a49bc: Generated new TSK 0
I20260811 12:52:00.341572 11241 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:52:00.347316 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 11090
W20260811 12:52:00.348517 11196 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.119:49537
W20260811 12:52:00.349299 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 11090: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:52:00.349465 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 11090
W20260811 12:52:00.376343 11066 connection.cc:535] client connection to 127.0.0.1:40459 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40459): Connection reset by peer (error 104)
2026-08-11T12:52:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2141 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:52:00 dist-test-slave-f0tn krb5kdc[11264](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:52:00 dist-test-slave-f0tn krb5kdc[11264](info): set up 2 sockets
Aug 11 12:52:00 dist-test-slave-f0tn krb5kdc[11264](info): commencing operation
krb5kdc: starting...
W20260811 12:52:01.637310 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:52:01 dist-test-slave-f0tn krb5kdc[11264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:52:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:52:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:52:01.966396 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32775
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:52:02.308837 11290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:52:02.309125 11290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:52:02.309262 11290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:52:02.316396 11290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:52:02.316507 11290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:52:02.316581 11290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:52:02.316661 11290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:52:02.316721 11290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:52:02.326610 11290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:52:02.327906 11290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:52:02.329241 11290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:52:02.340873 11299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:02.340903 11301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:02.340971 11297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:52:02.341643 11290 server_base.cc:1034] running on GCE node
I20260811 12:52:02.342489 11290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:52:02.343873 11290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:52:02.345078 11290 hybrid_clock.cc:648] HybridClock initialized: now 1786452722345033 us; error 34 us; skew 500 ppm
I20260811 12:52:02.345468 11290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:52:02 dist-test-slave-f0tn krb5kdc[11264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452722, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:52:02.348516 11290 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:52:02.349951 11290 webserver.cc:460] Webserver started at http://127.0.0.1:43669/ using document root <none> and password file <none>
I20260811 12:52:02.350610 11290 fs_manager.cc:359] Metadata directory not provided
I20260811 12:52:02.350735 11290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:52:02.351033 11290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:52:02.353986 11290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "a46e96fb0eeb4be691d556ccd3c4e970"
format_stamp: "Formatted at 2026-08-11 12:52:02 on dist-test-slave-f0tn"
server_key: "ebe1d142fd2eb8808f3c6022c32012ff"
server_key_iv: "29abd4fee7ad79dda5dffb24a7f6371a"
server_key_version: "encryptionkey@0"
I20260811 12:52:02.354725 11290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "a46e96fb0eeb4be691d556ccd3c4e970"
format_stamp: "Formatted at 2026-08-11 12:52:02 on dist-test-slave-f0tn"
server_key: "ebe1d142fd2eb8808f3c6022c32012ff"
server_key_iv: "29abd4fee7ad79dda5dffb24a7f6371a"
server_key_version: "encryptionkey@0"
I20260811 12:52:02.359635 11290 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:52:02.363070 11310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:52:02.364274 11290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:52:02.364434 11290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "a46e96fb0eeb4be691d556ccd3c4e970"
format_stamp: "Formatted at 2026-08-11 12:52:02 on dist-test-slave-f0tn"
server_key: "ebe1d142fd2eb8808f3c6022c32012ff"
server_key_iv: "29abd4fee7ad79dda5dffb24a7f6371a"
server_key_version: "encryptionkey@0"
I20260811 12:52:02.364606 11290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:52:02.369946 11290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:52:02.373752 11290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:52:02.374028 11290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:52:02.391312 11290 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32775
I20260811 12:52:02.391355 11409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32775 every 8 connection(s)
I20260811 12:52:02.392704 11290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:52:02.396554 11411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:52:02.398159 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 11290
I20260811 12:52:02.398375 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:52:02.398922 7613 external_mini_cluster.cc:1428] Setting key c1cbfb68d70492aaa5164a08e90a38d5
I20260811 12:52:02.406637 11415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:02.400815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:51521 (local address 127.0.0.1:32775)
0811 12:52:02.401631 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:02.401643 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:02.401696 (+ 53us) server_negotiation.cc:407] Connection header received
0811 12:52:02.403283 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:02.403309 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:02.403663 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:02.404011 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:02.405069 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:02.405075 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:02.405090 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:52:02.405104 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:02.405164 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:52:02.405185 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:02.405356 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:02.405504 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 12:52:02.405719 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":342,"threads_started":1}
I20260811 12:52:02.409976 11411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a46e96fb0eeb4be691d556ccd3c4e970: Bootstrap starting.
I20260811 12:52:02.410226 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:52:02.413723 11411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a46e96fb0eeb4be691d556ccd3c4e970: Neither blocks nor log segments found. Creating new log.
I20260811 12:52:02.415021 11411 log.cc:824] T 00000000000000000000000000000000 P a46e96fb0eeb4be691d556ccd3c4e970: Log is configured to *not* fsync() on all Append() calls
I20260811 12:52:02.417649 11415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:02.415298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:43997 (local address 127.0.0.1:32775)
0811 12:52:02.415483 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:02.415489 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:02.415931 (+ 442us) server_negotiation.cc:407] Connection header received
0811 12:52:02.416053 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:02.416059 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:02.416119 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:02.416231 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:02.417022 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:02.417026 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:02.417029 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:52:02.417043 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:02.417078 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:52:02.417092 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:02.417174 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:02.417299 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 12:52:02.417350 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:52:02.418365 11411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a46e96fb0eeb4be691d556ccd3c4e970: No bootstrap required, opened a new log
I20260811 12:52:02.425607 11411 raft_consensus.cc:348] T 00000000000000000000000000000000 P a46e96fb0eeb4be691d556ccd3c4e970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46e96fb0eeb4be691d556ccd3c4e970" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32775 } }
I20260811 12:52:02.425947 11411 raft_consensus.cc:374] T 00000000000000000000000000000000 P a46e96fb0eeb4be691d556ccd3c4e970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:52:02.426041 11411 raft_consensus.cc:729] T 00000000000000000000000000000000 P a46e96fb0eeb4be691d556ccd3c4e970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a46e96fb0eeb4be691d556ccd3c4e970, State: Initialized, Role: FOLLOWER
I20260811 12:52:02.426608 11411 consensus_queue.cc:260] T 00000000000000000000000000000000 P a46e96fb0eeb4be691d556ccd3c4e970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46e96fb0eeb4be691d556ccd3c4e970" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32775 } }
I20260811 12:52:02.426842 11411 raft_consensus.cc:388] T 00000000000000000000000000000000 P a46e96fb0eeb4be691d556ccd3c4e970 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:52:02.426971 11411 raft_consensus.cc:482] T 00000000000000000000000000000000 P a46e96fb0eeb4be691d556ccd3c4e970 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:52:02.427122 11411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a46e96fb0eeb4be691d556ccd3c4e970 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:52:02.429178 11411 raft_consensus.cc:504] T 00000000000000000000000000000000 P a46e96fb0eeb4be691d556ccd3c4e970 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46e96fb0eeb4be691d556ccd3c4e970" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32775 } }
I20260811 12:52:02.429664 11411 leader_election.cc:302] T 00000000000000000000000000000000 P a46e96fb0eeb4be691d556ccd3c4e970 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a46e96fb0eeb4be691d556ccd3c4e970; no voters:
I20260811 12:52:02.430210 11411 leader_election.cc:288] T 00000000000000000000000000000000 P a46e96fb0eeb4be691d556ccd3c4e970 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:52:02.430335 11427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a46e96fb0eeb4be691d556ccd3c4e970 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:52:02.430943 11427 raft_consensus.cc:686] T 00000000000000000000000000000000 P a46e96fb0eeb4be691d556ccd3c4e970 [term 1 LEADER]: Becoming Leader. State: Replica: a46e96fb0eeb4be691d556ccd3c4e970, State: Running, Role: LEADER
I20260811 12:52:02.431569 11427 consensus_queue.cc:237] T 00000000000000000000000000000000 P a46e96fb0eeb4be691d556ccd3c4e970 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46e96fb0eeb4be691d556ccd3c4e970" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32775 } }
I20260811 12:52:02.432226 11411 sys_catalog.cc:564] T 00000000000000000000000000000000 P a46e96fb0eeb4be691d556ccd3c4e970 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:52:02.434422 11429 sys_catalog.cc:455] T 00000000000000000000000000000000 P a46e96fb0eeb4be691d556ccd3c4e970 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a46e96fb0eeb4be691d556ccd3c4e970" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46e96fb0eeb4be691d556ccd3c4e970" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32775 } } }
I20260811 12:52:02.434494 11431 sys_catalog.cc:455] T 00000000000000000000000000000000 P a46e96fb0eeb4be691d556ccd3c4e970 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a46e96fb0eeb4be691d556ccd3c4e970. Latest consensus state: current_term: 1 leader_uuid: "a46e96fb0eeb4be691d556ccd3c4e970" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46e96fb0eeb4be691d556ccd3c4e970" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32775 } } }
I20260811 12:52:02.434814 11429 sys_catalog.cc:458] T 00000000000000000000000000000000 P a46e96fb0eeb4be691d556ccd3c4e970 [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:02.434839 11431 sys_catalog.cc:458] T 00000000000000000000000000000000 P a46e96fb0eeb4be691d556ccd3c4e970 [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:02.435683 11441 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:52:02.441859 11441 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:52:02.450337 11441 catalog_manager.cc:1347] Generated new cluster ID: b76b687a9a3840c0ba43c85910a5d15c
I20260811 12:52:02.450450 11441 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:52:02.474787 11441 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:52:02.475204 11441 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:52:02.483865 11441 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a46e96fb0eeb4be691d556ccd3c4e970: Generated new TSK 0
I20260811 12:52:02.484632 11441 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:52:02.498926 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 11290
W20260811 12:52:02.500180 11404 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.119:51521
W20260811 12:52:02.500815 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 11290: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:52:02.500941 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 11290
2026-08-11T12:52:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2167 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:52:02 dist-test-slave-f0tn krb5kdc[11463](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:52:02 dist-test-slave-f0tn krb5kdc[11463](info): set up 2 sockets
Aug 11 12:52:02 dist-test-slave-f0tn krb5kdc[11463](info): commencing operation
krb5kdc: starting...
W20260811 12:52:03.780385 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:52:03 dist-test-slave-f0tn krb5kdc[11463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:52:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:52:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:52:04.122136 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37239
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:52:04.484117 11488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:52:04.484414 11488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:52:04.484534 11488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:52:04.492146 11488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:52:04.492264 11488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:52:04.492339 11488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:52:04.492424 11488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:52:04.492486 11488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:52:04.502403 11488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37239
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:52:04.503718 11488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:52:04.505059 11488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:52:04.517936 11495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:04.518581 11502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:04.517838 11496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:52:04.518957 11488 server_base.cc:1034] running on GCE node
I20260811 12:52:04.519946 11488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:52:04.521481 11488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:52:04.522696 11488 hybrid_clock.cc:648] HybridClock initialized: now 1786452724522626 us; error 54 us; skew 500 ppm
I20260811 12:52:04.523041 11488 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:52:04 dist-test-slave-f0tn krb5kdc[11463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452724, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:52:04.526284 11488 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:52:04.527676 11488 webserver.cc:460] Webserver started at http://127.0.0.1:39649/ using document root <none> and password file <none>
I20260811 12:52:04.528348 11488 fs_manager.cc:359] Metadata directory not provided
I20260811 12:52:04.528445 11488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:52:04.528690 11488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:52:04.531623 11488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "3bb63c1e91894534a6322f6215a4ae10"
format_stamp: "Formatted at 2026-08-11 12:52:04 on dist-test-slave-f0tn"
server_key: "503d47639a88b7b9baba96ef35fc43bc"
server_key_iv: "b67d3e978170da52a1e19b44b38db6b8"
server_key_version: "encryptionkey@0"
I20260811 12:52:04.532321 11488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "3bb63c1e91894534a6322f6215a4ae10"
format_stamp: "Formatted at 2026-08-11 12:52:04 on dist-test-slave-f0tn"
server_key: "503d47639a88b7b9baba96ef35fc43bc"
server_key_iv: "b67d3e978170da52a1e19b44b38db6b8"
server_key_version: "encryptionkey@0"
I20260811 12:52:04.537228 11488 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:52:04.540539 11511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:52:04.541877 11488 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:52:04.542011 11488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "3bb63c1e91894534a6322f6215a4ae10"
format_stamp: "Formatted at 2026-08-11 12:52:04 on dist-test-slave-f0tn"
server_key: "503d47639a88b7b9baba96ef35fc43bc"
server_key_iv: "b67d3e978170da52a1e19b44b38db6b8"
server_key_version: "encryptionkey@0"
I20260811 12:52:04.542172 11488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:52:04.562530 11488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:52:04.566696 11488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:52:04.566933 11488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:52:04.584468 11488 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37239
I20260811 12:52:04.584504 11613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37239 every 8 connection(s)
I20260811 12:52:04.585856 11488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:52:04.589658 11614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:52:04.591753 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 11488
I20260811 12:52:04.591941 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:52:04.592481 7613 external_mini_cluster.cc:1428] Setting key 7a176d49b0a29d939090bcc51fd66996
I20260811 12:52:04.603276 11614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bb63c1e91894534a6322f6215a4ae10: Bootstrap starting.
Aug 11 12:52:04 dist-test-slave-f0tn krb5kdc[11463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 12:52:04.606751 11614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bb63c1e91894534a6322f6215a4ae10: Neither blocks nor log segments found. Creating new log.
I20260811 12:52:04.608033 11614 log.cc:824] T 00000000000000000000000000000000 P 3bb63c1e91894534a6322f6215a4ae10: Log is configured to *not* fsync() on all Append() calls
I20260811 12:52:04.611209 11614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bb63c1e91894534a6322f6215a4ae10: No bootstrap required, opened a new log
I20260811 12:52:04.614452 11618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:04.594529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:44417 (local address 127.0.0.1:37239)
0811 12:52:04.595324 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:04.595339 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:04.595384 (+ 45us) server_negotiation.cc:407] Connection header received
0811 12:52:04.596973 (+ 1589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:04.597000 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:04.597454 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:04.597817 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:04.599256 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:04.600588 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:04.601907 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:04.602575 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:04.602739 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:04.606999 (+ 4260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:04.607008 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:04.607022 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:04.607060 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:04.610208 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:04.611027 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:04.611033 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:04.611040 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:04.611215 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:04.611699 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:04.611703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:04.611705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:04.612224 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:04.612538 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:04.613125 (+ 587us) server_negotiation.cc:299] Negotiation successful
0811 12:52:04.613441 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":335,"threads_started":1}
I20260811 12:52:04.618320 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:52:04.619710 11614 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bb63c1e91894534a6322f6215a4ae10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bb63c1e91894534a6322f6215a4ae10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37239 } }
I20260811 12:52:04.620101 11614 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3bb63c1e91894534a6322f6215a4ae10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:52:04.620206 11614 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bb63c1e91894534a6322f6215a4ae10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb63c1e91894534a6322f6215a4ae10, State: Initialized, Role: FOLLOWER
I20260811 12:52:04.620843 11614 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bb63c1e91894534a6322f6215a4ae10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bb63c1e91894534a6322f6215a4ae10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37239 } }
I20260811 12:52:04.621129 11614 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3bb63c1e91894534a6322f6215a4ae10 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:52:04.621291 11614 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3bb63c1e91894534a6322f6215a4ae10 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:52:04.621537 11614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bb63c1e91894534a6322f6215a4ae10 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:52:04.624343 11614 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3bb63c1e91894534a6322f6215a4ae10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bb63c1e91894534a6322f6215a4ae10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37239 } }
I20260811 12:52:04.624883 11614 leader_election.cc:302] T 00000000000000000000000000000000 P 3bb63c1e91894534a6322f6215a4ae10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bb63c1e91894534a6322f6215a4ae10; no voters:
I20260811 12:52:04.625643 11614 leader_election.cc:288] T 00000000000000000000000000000000 P 3bb63c1e91894534a6322f6215a4ae10 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:52:04.625718 11632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3bb63c1e91894534a6322f6215a4ae10 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:52:04.628021 11614 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bb63c1e91894534a6322f6215a4ae10 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:52:04.629915 11632 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3bb63c1e91894534a6322f6215a4ae10 [term 1 LEADER]: Becoming Leader. State: Replica: 3bb63c1e91894534a6322f6215a4ae10, State: Running, Role: LEADER
I20260811 12:52:04.630647 11632 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bb63c1e91894534a6322f6215a4ae10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bb63c1e91894534a6322f6215a4ae10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37239 } }
I20260811 12:52:04.633695 11635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bb63c1e91894534a6322f6215a4ae10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bb63c1e91894534a6322f6215a4ae10. Latest consensus state: current_term: 1 leader_uuid: "3bb63c1e91894534a6322f6215a4ae10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bb63c1e91894534a6322f6215a4ae10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37239 } } }
I20260811 12:52:04.634059 11635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bb63c1e91894534a6322f6215a4ae10 [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:04.634598 11633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bb63c1e91894534a6322f6215a4ae10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3bb63c1e91894534a6322f6215a4ae10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bb63c1e91894534a6322f6215a4ae10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37239 } } }
I20260811 12:52:04.634900 11633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bb63c1e91894534a6322f6215a4ae10 [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:04.636350 11644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:52:04.641646 11618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:04.622702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:43201 (local address 127.0.0.1:37239)
0811 12:52:04.622896 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:04.622902 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:04.623205 (+ 303us) server_negotiation.cc:407] Connection header received
0811 12:52:04.623343 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:04.623349 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:04.623435 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:04.623562 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:04.625015 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:04.626321 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:04.628243 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:04.628934 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:04.629095 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:04.632283 (+ 3188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:04.632291 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:04.632297 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:04.632348 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:04.636716 (+ 4368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:04.637661 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:04.637667 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:04.637670 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:04.637861 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:04.638622 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:04.638629 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:04.638633 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:04.639134 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:04.640873 (+ 1739us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:04.641064 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 12:52:04.641259 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:52:04.643006 11644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:52:04.652365 11644 catalog_manager.cc:1347] Generated new cluster ID: e4f247e0d2cb48dca11cdd16774df017
I20260811 12:52:04.652498 11644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:52:04.675980 11644 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:52:04.677924 11644 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:52:04.688634 11644 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3bb63c1e91894534a6322f6215a4ae10: Generated new TSK 0
I20260811 12:52:04.689678 11644 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:52:04.704219 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 11488
I20260811 12:52:04.705494 11606 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:52:04.796335 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 11488
W20260811 12:52:04.819964 11459 connection.cc:535] client connection to 127.0.0.1:37239 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37239): Connection reset by peer (error 104)
2026-08-11T12:52:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2275 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:52:04 dist-test-slave-f0tn krb5kdc[11668](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:52:04 dist-test-slave-f0tn krb5kdc[11668](info): set up 2 sockets
Aug 11 12:52:04 dist-test-slave-f0tn krb5kdc[11668](info): commencing operation
krb5kdc: starting...
W20260811 12:52:06.050364 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.001s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:52:06 dist-test-slave-f0tn krb5kdc[11668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:52:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:52:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:52:06.401283 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45919
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:52:06.752513 11692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:52:06.752805 11692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:52:06.752926 11692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:52:06.760056 11692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:52:06.760192 11692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:52:06.760267 11692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:52:06.760351 11692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:52:06.760413 11692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:52:06.770756 11692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45919
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:52:06.772130 11692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:52:06.773550 11692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:52:06.785454 11700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:06.785542 11699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:06.785568 11704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:52:06.786336 11692 server_base.cc:1034] running on GCE node
I20260811 12:52:06.787213 11692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:52:06.788738 11692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:52:06.789950 11692 hybrid_clock.cc:648] HybridClock initialized: now 1786452726789895 us; error 48 us; skew 500 ppm
I20260811 12:52:06.790316 11692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:52:06 dist-test-slave-f0tn krb5kdc[11668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452726, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:52:06.793545 11692 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:52:06.795074 11692 webserver.cc:460] Webserver started at http://127.0.0.1:39401/ using document root <none> and password file <none>
I20260811 12:52:06.795804 11692 fs_manager.cc:359] Metadata directory not provided
I20260811 12:52:06.795926 11692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:52:06.796255 11692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:52:06.799418 11692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "ea6a77afe8474606837037e662dadc69"
format_stamp: "Formatted at 2026-08-11 12:52:06 on dist-test-slave-f0tn"
server_key: "ba2e907438c906982b77d97d35b51957"
server_key_iv: "4713b88500470297bca3f1639868797f"
server_key_version: "encryptionkey@0"
I20260811 12:52:06.800199 11692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "ea6a77afe8474606837037e662dadc69"
format_stamp: "Formatted at 2026-08-11 12:52:06 on dist-test-slave-f0tn"
server_key: "ba2e907438c906982b77d97d35b51957"
server_key_iv: "4713b88500470297bca3f1639868797f"
server_key_version: "encryptionkey@0"
I20260811 12:52:06.805612 11692 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:52:06.809237 11716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:52:06.810721 11692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:52:06.810907 11692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "ea6a77afe8474606837037e662dadc69"
format_stamp: "Formatted at 2026-08-11 12:52:06 on dist-test-slave-f0tn"
server_key: "ba2e907438c906982b77d97d35b51957"
server_key_iv: "4713b88500470297bca3f1639868797f"
server_key_version: "encryptionkey@0"
I20260811 12:52:06.811117 11692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:52:06.856237 11692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:52:06.860561 11692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:52:06.860863 11692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:52:06.878641 11692 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45919
I20260811 12:52:06.878660 11807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45919 every 8 connection(s)
I20260811 12:52:06.879998 11692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:52:06.883718 11808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:52:06.885721 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 11692
I20260811 12:52:06.885911 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:52:06.886453 7613 external_mini_cluster.cc:1428] Setting key 9004ba5e12e32cb2015df3571f9f337d
I20260811 12:52:06.895744 11808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69: Bootstrap starting.
I20260811 12:52:06.899268 11808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69: Neither blocks nor log segments found. Creating new log.
I20260811 12:52:06.899746 11814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:06.888280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:60811 (local address 127.0.0.1:45919)
0811 12:52:06.889054 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:06.889066 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:06.889118 (+ 52us) server_negotiation.cc:407] Connection header received
0811 12:52:06.890830 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:06.890855 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:06.891238 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:06.891545 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:06.892199 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:06.893589 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:06.895335 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:06.896037 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:06.896168 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:06.897474 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:06.897483 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:06.897497 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:52:06.898019 (+ 522us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:06.898096 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:52:06.898121 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:06.898255 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:06.898528 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 12:52:06.898697 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":337,"threads_started":1}
I20260811 12:52:06.900395 11808 log.cc:824] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69: Log is configured to *not* fsync() on all Append() calls
I20260811 12:52:06.903400 11808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69: No bootstrap required, opened a new log
I20260811 12:52:06.903580 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:52:06.911491 11808 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6a77afe8474606837037e662dadc69" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45919 } }
I20260811 12:52:06.911916 11808 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:52:06.912019 11808 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea6a77afe8474606837037e662dadc69, State: Initialized, Role: FOLLOWER
I20260811 12:52:06.912693 11808 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6a77afe8474606837037e662dadc69" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45919 } }
I20260811 12:52:06.912956 11808 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:52:06.913072 11808 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:52:06.913188 11808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:52:06.914369 11814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:06.907977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:40449 (local address 127.0.0.1:45919)
0811 12:52:06.908157 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:06.908162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:06.908583 (+ 421us) server_negotiation.cc:407] Connection header received
0811 12:52:06.908747 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:06.908753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:06.908832 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:06.908932 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:06.909409 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:06.910552 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:06.911784 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:06.912418 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:06.912536 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:06.913583 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:06.913588 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:06.913593 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:52:06.913613 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:06.913681 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:52:06.913700 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:06.913838 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:06.913978 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 12:52:06.914029 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:52:06.916101 11808 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6a77afe8474606837037e662dadc69" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45919 } }
I20260811 12:52:06.916556 11808 leader_election.cc:302] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea6a77afe8474606837037e662dadc69; no voters:
I20260811 12:52:06.917104 11808 leader_election.cc:288] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:52:06.917235 11828 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:52:06.919068 11808 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:52:06.921640 11828 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69 [term 1 LEADER]: Becoming Leader. State: Replica: ea6a77afe8474606837037e662dadc69, State: Running, Role: LEADER
I20260811 12:52:06.922223 11828 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6a77afe8474606837037e662dadc69" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45919 } }
I20260811 12:52:06.924908 11831 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea6a77afe8474606837037e662dadc69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6a77afe8474606837037e662dadc69" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45919 } } }
I20260811 12:52:06.925299 11831 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69 [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:06.925683 11833 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea6a77afe8474606837037e662dadc69. Latest consensus state: current_term: 1 leader_uuid: "ea6a77afe8474606837037e662dadc69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6a77afe8474606837037e662dadc69" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45919 } } }
I20260811 12:52:06.925994 11833 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69 [sys.catalog]: This master's current role is: LEADER
W20260811 12:52:06.931550 11853 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:52:06.931655 11853 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:52:06.931762 11856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:52:06.933784 11856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:52:06.941331 11856 catalog_manager.cc:1347] Generated new cluster ID: 73c114a3cb2c4deb8a80edfa6214dd4d
I20260811 12:52:06.941457 11856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:52:06.952971 11856 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:52:06.954424 11856 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:52:06.962366 11856 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea6a77afe8474606837037e662dadc69: Generated new TSK 0
I20260811 12:52:06.963168 11856 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:52:06.973006 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 11692
W20260811 12:52:06.974198 11801 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.119:60811
W20260811 12:52:06.975005 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 11692: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:52:06.975165 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 11692
W20260811 12:52:07.001477 11663 connection.cc:535] client connection to 127.0.0.1:45919 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45919): Connection reset by peer (error 104)
2026-08-11T12:52:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2181 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:52:07 dist-test-slave-f0tn krb5kdc[11871](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:52:07 dist-test-slave-f0tn krb5kdc[11871](info): set up 2 sockets
Aug 11 12:52:07 dist-test-slave-f0tn krb5kdc[11871](info): commencing operation
krb5kdc: starting...
W20260811 12:52:08.281225 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.261s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:52:08 dist-test-slave-f0tn krb5kdc[11871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:52:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:52:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:52:08.616506 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44607
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:52:08.959658 11897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:52:08.959937 11897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:52:08.960071 11897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:52:08.967109 11897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:52:08.967214 11897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:52:08.967294 11897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:52:08.967378 11897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:52:08.967440 11897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:52:08.976943 11897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44607
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:52:08.978168 11897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:52:08.979425 11897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:52:08.990803 11907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:08.990863 11902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:08.990650 11905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:52:08.991930 11897 server_base.cc:1034] running on GCE node
I20260811 12:52:08.992772 11897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:52:08.994241 11897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:52:08.995473 11897 hybrid_clock.cc:648] HybridClock initialized: now 1786452728995413 us; error 35 us; skew 500 ppm
I20260811 12:52:08.995905 11897 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:52:08 dist-test-slave-f0tn krb5kdc[11871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452728, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:52:08.999090 11897 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:52:09.000510 11897 webserver.cc:460] Webserver started at http://127.0.0.1:33495/ using document root <none> and password file <none>
I20260811 12:52:09.001184 11897 fs_manager.cc:359] Metadata directory not provided
I20260811 12:52:09.001314 11897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:52:09.001658 11897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:52:09.004771 11897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "ace6745ac7164e22b470dbd95a10975a"
format_stamp: "Formatted at 2026-08-11 12:52:08 on dist-test-slave-f0tn"
server_key: "379b60ecf17deae919e994da605dbd6b"
server_key_iv: "c6653c99d8b564aad8aeccf1af7173e9"
server_key_version: "encryptionkey@0"
I20260811 12:52:09.005635 11897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "ace6745ac7164e22b470dbd95a10975a"
format_stamp: "Formatted at 2026-08-11 12:52:08 on dist-test-slave-f0tn"
server_key: "379b60ecf17deae919e994da605dbd6b"
server_key_iv: "c6653c99d8b564aad8aeccf1af7173e9"
server_key_version: "encryptionkey@0"
I20260811 12:52:09.010848 11897 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 12:52:09.014365 11918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:52:09.015645 11897 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:52:09.015825 11897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "ace6745ac7164e22b470dbd95a10975a"
format_stamp: "Formatted at 2026-08-11 12:52:08 on dist-test-slave-f0tn"
server_key: "379b60ecf17deae919e994da605dbd6b"
server_key_iv: "c6653c99d8b564aad8aeccf1af7173e9"
server_key_version: "encryptionkey@0"
I20260811 12:52:09.016021 11897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:52:09.021865 11897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:52:09.025641 11897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:52:09.025916 11897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:52:09.043706 11897 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44607
I20260811 12:52:09.043774 11997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44607 every 8 connection(s)
I20260811 12:52:09.045099 11897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:52:09.048888 11999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:52:09.051033 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 11897
I20260811 12:52:09.051219 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:52:09.051760 7613 external_mini_cluster.cc:1428] Setting key 1db14ac6db57c0c333c3bef04a779741
I20260811 12:52:09.059466 12003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:09.053664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:34645 (local address 127.0.0.1:44607)
0811 12:52:09.054430 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:09.054442 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:09.054484 (+ 42us) server_negotiation.cc:407] Connection header received
0811 12:52:09.056112 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:09.056133 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:09.056463 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:09.056806 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:09.057890 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:09.057896 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:09.057909 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:52:09.057924 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:09.057991 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:52:09.058018 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:09.058275 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:09.058432 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 12:52:09.058600 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":314,"threads_started":1}
I20260811 12:52:09.060483 11999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ace6745ac7164e22b470dbd95a10975a: Bootstrap starting.
I20260811 12:52:09.063439 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:52:09.064143 11999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ace6745ac7164e22b470dbd95a10975a: Neither blocks nor log segments found. Creating new log.
I20260811 12:52:09.065107 11999 log.cc:824] T 00000000000000000000000000000000 P ace6745ac7164e22b470dbd95a10975a: Log is configured to *not* fsync() on all Append() calls
I20260811 12:52:09.067889 11999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ace6745ac7164e22b470dbd95a10975a: No bootstrap required, opened a new log
I20260811 12:52:09.070006 12003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:09.067729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:38155 (local address 127.0.0.1:44607)
0811 12:52:09.067893 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:09.067898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:09.068254 (+ 356us) server_negotiation.cc:407] Connection header received
0811 12:52:09.068379 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:09.068385 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:09.068449 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:09.068556 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:09.069403 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:09.069409 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:09.069414 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:52:09.069434 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:09.069480 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:52:09.069500 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:09.069625 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:09.069675 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 12:52:09.069732 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:52:09.076042 11999 raft_consensus.cc:348] T 00000000000000000000000000000000 P ace6745ac7164e22b470dbd95a10975a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ace6745ac7164e22b470dbd95a10975a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44607 } }
I20260811 12:52:09.076355 11999 raft_consensus.cc:374] T 00000000000000000000000000000000 P ace6745ac7164e22b470dbd95a10975a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:52:09.076426 11999 raft_consensus.cc:729] T 00000000000000000000000000000000 P ace6745ac7164e22b470dbd95a10975a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace6745ac7164e22b470dbd95a10975a, State: Initialized, Role: FOLLOWER
I20260811 12:52:09.076906 11999 consensus_queue.cc:260] T 00000000000000000000000000000000 P ace6745ac7164e22b470dbd95a10975a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ace6745ac7164e22b470dbd95a10975a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44607 } }
I20260811 12:52:09.077098 11999 raft_consensus.cc:388] T 00000000000000000000000000000000 P ace6745ac7164e22b470dbd95a10975a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:52:09.077191 11999 raft_consensus.cc:482] T 00000000000000000000000000000000 P ace6745ac7164e22b470dbd95a10975a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:52:09.077307 11999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ace6745ac7164e22b470dbd95a10975a [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:52:09.079362 11999 raft_consensus.cc:504] T 00000000000000000000000000000000 P ace6745ac7164e22b470dbd95a10975a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ace6745ac7164e22b470dbd95a10975a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44607 } }
I20260811 12:52:09.079792 11999 leader_election.cc:302] T 00000000000000000000000000000000 P ace6745ac7164e22b470dbd95a10975a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ace6745ac7164e22b470dbd95a10975a; no voters:
I20260811 12:52:09.080298 11999 leader_election.cc:288] T 00000000000000000000000000000000 P ace6745ac7164e22b470dbd95a10975a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:52:09.080430 12020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ace6745ac7164e22b470dbd95a10975a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:52:09.081555 12020 raft_consensus.cc:686] T 00000000000000000000000000000000 P ace6745ac7164e22b470dbd95a10975a [term 1 LEADER]: Becoming Leader. State: Replica: ace6745ac7164e22b470dbd95a10975a, State: Running, Role: LEADER
I20260811 12:52:09.082178 12020 consensus_queue.cc:237] T 00000000000000000000000000000000 P ace6745ac7164e22b470dbd95a10975a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ace6745ac7164e22b470dbd95a10975a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44607 } }
I20260811 12:52:09.082293 11999 sys_catalog.cc:564] T 00000000000000000000000000000000 P ace6745ac7164e22b470dbd95a10975a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:52:09.084941 12024 sys_catalog.cc:455] T 00000000000000000000000000000000 P ace6745ac7164e22b470dbd95a10975a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ace6745ac7164e22b470dbd95a10975a. Latest consensus state: current_term: 1 leader_uuid: "ace6745ac7164e22b470dbd95a10975a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ace6745ac7164e22b470dbd95a10975a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44607 } } }
I20260811 12:52:09.085532 12024 sys_catalog.cc:458] T 00000000000000000000000000000000 P ace6745ac7164e22b470dbd95a10975a [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:09.084966 12022 sys_catalog.cc:455] T 00000000000000000000000000000000 P ace6745ac7164e22b470dbd95a10975a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ace6745ac7164e22b470dbd95a10975a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ace6745ac7164e22b470dbd95a10975a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44607 } } }
I20260811 12:52:09.085961 12022 sys_catalog.cc:458] T 00000000000000000000000000000000 P ace6745ac7164e22b470dbd95a10975a [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:09.088655 12035 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:52:09.092641 12035 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:52:09.101194 12035 catalog_manager.cc:1347] Generated new cluster ID: b9d0233e0e9d4a81aecaac09918555ed
I20260811 12:52:09.101321 12035 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:52:09.115760 12035 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:52:09.116299 12035 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:52:09.139794 12035 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ace6745ac7164e22b470dbd95a10975a: Generated new TSK 0
I20260811 12:52:09.140751 12035 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:52:09.147024 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 11897
W20260811 12:52:09.148147 11989 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.119:34645
W20260811 12:52:09.148837 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 11897: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:52:09.148978 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 11897
2026-08-11T12:52:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2177 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:52:09 dist-test-slave-f0tn krb5kdc[12059](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:52:09 dist-test-slave-f0tn krb5kdc[12059](info): set up 2 sockets
Aug 11 12:52:09 dist-test-slave-f0tn krb5kdc[12059](info): commencing operation
krb5kdc: starting...
W20260811 12:52:10.430516 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:52:10 dist-test-slave-f0tn krb5kdc[12059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:52:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:52:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:52:10.763659 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:38123
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:40855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:38123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:52:11.100795 12081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:52:11.101111 12081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:52:11.101249 12081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:52:11.108493 12081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:52:11.108605 12081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:52:11.108681 12081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:52:11.108760 12081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:52:11.108819 12081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:52:11.118724 12081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:40855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:38123
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:38123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:52:11.120008 12081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:52:11.121345 12081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:52:11.132810 12088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:11.132774 12087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:11.133278 12090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:52:11.133682 12081 server_base.cc:1034] running on GCE node
I20260811 12:52:11.134465 12081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:52:11.135823 12081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:52:11.137027 12081 hybrid_clock.cc:648] HybridClock initialized: now 1786452731136973 us; error 48 us; skew 500 ppm
I20260811 12:52:11.137362 12081 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:52:11 dist-test-slave-f0tn krb5kdc[12059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452731, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:52:11.140443 12081 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:52:11.141949 12081 webserver.cc:460] Webserver started at http://127.7.111.126:34157/ using document root <none> and password file <none>
I20260811 12:52:11.142663 12081 fs_manager.cc:359] Metadata directory not provided
I20260811 12:52:11.142789 12081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:52:11.143093 12081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:52:11.146342 12081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "46f7ae41c7e94dd0ba6ed6493a3ede0d"
format_stamp: "Formatted at 2026-08-11 12:52:11 on dist-test-slave-f0tn"
server_key: "b347ec9b252421b0461c468e3ed7afd7"
server_key_iv: "41b5407a4f65e9c3d314728b792f6bb4"
server_key_version: "encryptionkey@0"
I20260811 12:52:11.147117 12081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "46f7ae41c7e94dd0ba6ed6493a3ede0d"
format_stamp: "Formatted at 2026-08-11 12:52:11 on dist-test-slave-f0tn"
server_key: "b347ec9b252421b0461c468e3ed7afd7"
server_key_iv: "41b5407a4f65e9c3d314728b792f6bb4"
server_key_version: "encryptionkey@0"
I20260811 12:52:11.152371 12081 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:52:11.155957 12107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:52:11.157361 12081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:52:11.157622 12081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "46f7ae41c7e94dd0ba6ed6493a3ede0d"
format_stamp: "Formatted at 2026-08-11 12:52:11 on dist-test-slave-f0tn"
server_key: "b347ec9b252421b0461c468e3ed7afd7"
server_key_iv: "41b5407a4f65e9c3d314728b792f6bb4"
server_key_version: "encryptionkey@0"
I20260811 12:52:11.157820 12081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:52:11.173112 12081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:52:11.177802 12081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:52:11.178085 12081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:52:11.195845 12081 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:38123
I20260811 12:52:11.195874 12216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:38123 every 8 connection(s)
I20260811 12:52:11.197238 12081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:52:11.200980 12220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:52:11.204584 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 12081
I20260811 12:52:11.204782 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:52:11.205292 7613 external_mini_cluster.cc:1428] Setting key 996dc6b10f0e0b9a6c366ca414fd85fd
I20260811 12:52:11.213338 12220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46f7ae41c7e94dd0ba6ed6493a3ede0d: Bootstrap starting.
I20260811 12:52:11.217024 12220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46f7ae41c7e94dd0ba6ed6493a3ede0d: Neither blocks nor log segments found. Creating new log.
I20260811 12:52:11.218379 12220 log.cc:824] T 00000000000000000000000000000000 P 46f7ae41c7e94dd0ba6ed6493a3ede0d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:52:11.221910 12220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46f7ae41c7e94dd0ba6ed6493a3ede0d: No bootstrap required, opened a new log
Aug 11 12:52:11 dist-test-slave-f0tn krb5kdc[12059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:52:11.232414 12226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:11.207273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45735 (local address 127.7.111.126:38123)
0811 12:52:11.208199 (+ 926us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:11.208215 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:11.208575 (+ 360us) server_negotiation.cc:407] Connection header received
0811 12:52:11.211549 (+ 2974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:11.211580 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:11.212099 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:11.212546 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:11.213998 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:11.215951 (+ 1953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:11.217215 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:11.218269 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:11.218467 (+ 198us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:11.223225 (+ 4758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:11.223238 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:11.223261 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:11.223317 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:11.227425 (+ 4108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:11.228312 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:11.228322 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:11.228333 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:11.228554 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:11.228975 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:11.228980 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:11.228983 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:11.229791 (+ 808us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:11.230141 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:11.230625 (+ 484us) server_negotiation.cc:299] Negotiation successful
0811 12:52:11.231017 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":434,"threads_started":1}
I20260811 12:52:11.232381 12220 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46f7ae41c7e94dd0ba6ed6493a3ede0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f7ae41c7e94dd0ba6ed6493a3ede0d" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38123 } }
I20260811 12:52:11.232903 12220 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46f7ae41c7e94dd0ba6ed6493a3ede0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:52:11.233026 12220 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46f7ae41c7e94dd0ba6ed6493a3ede0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46f7ae41c7e94dd0ba6ed6493a3ede0d, State: Initialized, Role: FOLLOWER
I20260811 12:52:11.233662 12220 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46f7ae41c7e94dd0ba6ed6493a3ede0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f7ae41c7e94dd0ba6ed6493a3ede0d" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38123 } }
I20260811 12:52:11.233903 12220 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46f7ae41c7e94dd0ba6ed6493a3ede0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:52:11.234040 12220 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46f7ae41c7e94dd0ba6ed6493a3ede0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:52:11.234218 12220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46f7ae41c7e94dd0ba6ed6493a3ede0d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:52:11.236495 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:52:11.236778 12220 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46f7ae41c7e94dd0ba6ed6493a3ede0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f7ae41c7e94dd0ba6ed6493a3ede0d" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38123 } }
I20260811 12:52:11.237344 12220 leader_election.cc:302] T 00000000000000000000000000000000 P 46f7ae41c7e94dd0ba6ed6493a3ede0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46f7ae41c7e94dd0ba6ed6493a3ede0d; no voters:
I20260811 12:52:11.237998 12220 leader_election.cc:288] T 00000000000000000000000000000000 P 46f7ae41c7e94dd0ba6ed6493a3ede0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:52:11.238103 12230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46f7ae41c7e94dd0ba6ed6493a3ede0d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:52:11.239233 12230 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46f7ae41c7e94dd0ba6ed6493a3ede0d [term 1 LEADER]: Becoming Leader. State: Replica: 46f7ae41c7e94dd0ba6ed6493a3ede0d, State: Running, Role: LEADER
I20260811 12:52:11.239854 12230 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46f7ae41c7e94dd0ba6ed6493a3ede0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f7ae41c7e94dd0ba6ed6493a3ede0d" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38123 } }
I20260811 12:52:11.240355 12220 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46f7ae41c7e94dd0ba6ed6493a3ede0d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:52:11.242722 12232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46f7ae41c7e94dd0ba6ed6493a3ede0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46f7ae41c7e94dd0ba6ed6493a3ede0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f7ae41c7e94dd0ba6ed6493a3ede0d" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38123 } } }
I20260811 12:52:11.243062 12232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46f7ae41c7e94dd0ba6ed6493a3ede0d [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:11.242455 12234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46f7ae41c7e94dd0ba6ed6493a3ede0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46f7ae41c7e94dd0ba6ed6493a3ede0d. Latest consensus state: current_term: 1 leader_uuid: "46f7ae41c7e94dd0ba6ed6493a3ede0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f7ae41c7e94dd0ba6ed6493a3ede0d" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 38123 } } }
I20260811 12:52:11.243813 12234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46f7ae41c7e94dd0ba6ed6493a3ede0d [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:11.249555 12243 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:52:11.256824 12243 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:52:11.267318 12243 catalog_manager.cc:1347] Generated new cluster ID: a216c0def3a748fdb5f1c5836054ff05
I20260811 12:52:11.267436 12243 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:52:11.284300 12243 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:52:11.286655 12243 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:52:11.296043 12243 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46f7ae41c7e94dd0ba6ed6493a3ede0d: Generated new TSK 0
I20260811 12:52:11.297261 12243 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:52:11.297783 12226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:11.261057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37041 (local address 127.7.111.126:38123)
0811 12:52:11.261275 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:11.261282 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:11.277998 (+ 16716us) server_negotiation.cc:407] Connection header received
0811 12:52:11.278147 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:11.278154 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:11.278257 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:11.278382 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:11.280832 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:11.282128 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:11.284122 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:11.284874 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:11.284982 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:11.288696 (+ 3714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:11.288704 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:11.288710 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:11.288751 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:11.292753 (+ 4002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:11.293964 (+ 1211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:11.293971 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:11.293975 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:11.294174 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:11.294788 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:11.294794 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:11.294798 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:11.295206 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:11.295487 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:11.296984 (+ 1497us) server_negotiation.cc:299] Negotiation successful
0811 12:52:11.297229 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":84}
I20260811 12:52:11.310807 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 12081
I20260811 12:52:11.311957 12209 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:52:11.402756 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 12081
W20260811 12:52:11.427932 12055 connection.cc:535] client connection to 127.7.111.126:38123 recv error: Network error: failed to read from TLS socket (remote: 127.7.111.126:38123): Connection reset by peer (error 104)
2026-08-11T12:52:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2248 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:52:11 dist-test-slave-f0tn krb5kdc[12281](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:52:11 dist-test-slave-f0tn krb5kdc[12281](info): set up 2 sockets
Aug 11 12:52:11 dist-test-slave-f0tn krb5kdc[12281](info): commencing operation
krb5kdc: starting...
W20260811 12:52:12.678963 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.000s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:52:12 dist-test-slave-f0tn krb5kdc[12281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:52:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:52:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:52:13.017973 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:41977
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:35837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:41977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:52:13.371765 12298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:52:13.372097 12298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:52:13.372256 12298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:52:13.379611 12298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:52:13.379758 12298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:52:13.379851 12298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:52:13.379948 12298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:52:13.380030 12298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:52:13.390311 12298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:35837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:41977
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:41977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:52:13.391757 12298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:52:13.393249 12298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:52:13.405898 12308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:13.405838 12312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:13.405886 12309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:52:13.406785 12298 server_base.cc:1034] running on GCE node
I20260811 12:52:13.407648 12298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:52:13.409125 12298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:52:13.410339 12298 hybrid_clock.cc:648] HybridClock initialized: now 1786452733410293 us; error 31 us; skew 500 ppm
I20260811 12:52:13.410699 12298 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:52:13 dist-test-slave-f0tn krb5kdc[12281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452733, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:52:13.413847 12298 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:52:13.415401 12298 webserver.cc:460] Webserver started at http://127.7.111.126:37251/ using document root <none> and password file <none>
I20260811 12:52:13.416106 12298 fs_manager.cc:359] Metadata directory not provided
I20260811 12:52:13.416239 12298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:52:13.416558 12298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:52:13.419760 12298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "f25610539c2a4334b08916bbe4683586"
format_stamp: "Formatted at 2026-08-11 12:52:13 on dist-test-slave-f0tn"
server_key: "3834e675b24307e037c733b041e9d950"
server_key_iv: "7e07c5582f1ac36d490e31f218f9b3e3"
server_key_version: "encryptionkey@0"
I20260811 12:52:13.420569 12298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "f25610539c2a4334b08916bbe4683586"
format_stamp: "Formatted at 2026-08-11 12:52:13 on dist-test-slave-f0tn"
server_key: "3834e675b24307e037c733b041e9d950"
server_key_iv: "7e07c5582f1ac36d490e31f218f9b3e3"
server_key_version: "encryptionkey@0"
I20260811 12:52:13.426028 12298 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:52:13.429831 12324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:52:13.431152 12298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:52:13.431337 12298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "f25610539c2a4334b08916bbe4683586"
format_stamp: "Formatted at 2026-08-11 12:52:13 on dist-test-slave-f0tn"
server_key: "3834e675b24307e037c733b041e9d950"
server_key_iv: "7e07c5582f1ac36d490e31f218f9b3e3"
server_key_version: "encryptionkey@0"
I20260811 12:52:13.431529 12298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:52:13.445348 12298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:52:13.449190 12298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:52:13.449494 12298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:52:13.466872 12298 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:41977
I20260811 12:52:13.466892 12418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:41977 every 8 connection(s)
I20260811 12:52:13.468263 12298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:52:13.470736 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 12298
I20260811 12:52:13.470894 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:52:13.471325 7613 external_mini_cluster.cc:1428] Setting key 121ecc5f98692dca1ded199a6bc3f37a
I20260811 12:52:13.472051 12419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:52:13.484345 12423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:13.473055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32803 (local address 127.7.111.126:41977)
0811 12:52:13.473825 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:13.473837 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:13.473878 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:52:13.475397 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:13.475418 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:13.475802 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:13.476129 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:13.476683 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:13.478054 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:13.479890 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:13.480551 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:13.480676 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:13.482051 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:13.482059 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:13.482076 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:52:13.482652 (+ 576us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:13.482732 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:52:13.482758 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:13.482888 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:13.483160 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 12:52:13.483365 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":357,"threads_started":1}
I20260811 12:52:13.485476 12419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f25610539c2a4334b08916bbe4683586: Bootstrap starting.
I20260811 12:52:13.487988 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:52:13.488720 12419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f25610539c2a4334b08916bbe4683586: Neither blocks nor log segments found. Creating new log.
I20260811 12:52:13.489982 12419 log.cc:824] T 00000000000000000000000000000000 P f25610539c2a4334b08916bbe4683586: Log is configured to *not* fsync() on all Append() calls
I20260811 12:52:13.492897 12419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f25610539c2a4334b08916bbe4683586: No bootstrap required, opened a new log
I20260811 12:52:13.499275 12423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:13.493061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47411 (local address 127.7.111.126:41977)
0811 12:52:13.493241 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:13.493247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:13.493560 (+ 313us) server_negotiation.cc:407] Connection header received
0811 12:52:13.493723 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:13.493729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:13.493809 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:13.493917 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:13.494334 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:13.495474 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:13.496705 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:13.497355 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:13.497497 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:13.498500 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:13.498507 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:13.498512 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:52:13.498532 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:13.498583 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:52:13.498602 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:13.498760 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:13.498880 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 12:52:13.498933 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 12:52:13.502290 12419 raft_consensus.cc:348] T 00000000000000000000000000000000 P f25610539c2a4334b08916bbe4683586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25610539c2a4334b08916bbe4683586" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 41977 } }
I20260811 12:52:13.502630 12419 raft_consensus.cc:374] T 00000000000000000000000000000000 P f25610539c2a4334b08916bbe4683586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:52:13.502703 12419 raft_consensus.cc:729] T 00000000000000000000000000000000 P f25610539c2a4334b08916bbe4683586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f25610539c2a4334b08916bbe4683586, State: Initialized, Role: FOLLOWER
I20260811 12:52:13.503206 12419 consensus_queue.cc:260] T 00000000000000000000000000000000 P f25610539c2a4334b08916bbe4683586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25610539c2a4334b08916bbe4683586" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 41977 } }
I20260811 12:52:13.503407 12419 raft_consensus.cc:388] T 00000000000000000000000000000000 P f25610539c2a4334b08916bbe4683586 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:52:13.503501 12419 raft_consensus.cc:482] T 00000000000000000000000000000000 P f25610539c2a4334b08916bbe4683586 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:52:13.503621 12419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f25610539c2a4334b08916bbe4683586 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:52:13.505754 12419 raft_consensus.cc:504] T 00000000000000000000000000000000 P f25610539c2a4334b08916bbe4683586 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25610539c2a4334b08916bbe4683586" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 41977 } }
I20260811 12:52:13.506229 12419 leader_election.cc:302] T 00000000000000000000000000000000 P f25610539c2a4334b08916bbe4683586 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f25610539c2a4334b08916bbe4683586; no voters:
I20260811 12:52:13.506886 12419 leader_election.cc:288] T 00000000000000000000000000000000 P f25610539c2a4334b08916bbe4683586 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:52:13.506976 12437 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f25610539c2a4334b08916bbe4683586 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:52:13.507710 12437 raft_consensus.cc:686] T 00000000000000000000000000000000 P f25610539c2a4334b08916bbe4683586 [term 1 LEADER]: Becoming Leader. State: Replica: f25610539c2a4334b08916bbe4683586, State: Running, Role: LEADER
I20260811 12:52:13.508214 12437 consensus_queue.cc:237] T 00000000000000000000000000000000 P f25610539c2a4334b08916bbe4683586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25610539c2a4334b08916bbe4683586" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 41977 } }
I20260811 12:52:13.508920 12419 sys_catalog.cc:564] T 00000000000000000000000000000000 P f25610539c2a4334b08916bbe4683586 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:52:13.510725 12439 sys_catalog.cc:455] T 00000000000000000000000000000000 P f25610539c2a4334b08916bbe4683586 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f25610539c2a4334b08916bbe4683586" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25610539c2a4334b08916bbe4683586" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 41977 } } }
I20260811 12:52:13.511106 12439 sys_catalog.cc:458] T 00000000000000000000000000000000 P f25610539c2a4334b08916bbe4683586 [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:13.511118 12440 sys_catalog.cc:455] T 00000000000000000000000000000000 P f25610539c2a4334b08916bbe4683586 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f25610539c2a4334b08916bbe4683586. Latest consensus state: current_term: 1 leader_uuid: "f25610539c2a4334b08916bbe4683586" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25610539c2a4334b08916bbe4683586" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 41977 } } }
I20260811 12:52:13.511781 12440 sys_catalog.cc:458] T 00000000000000000000000000000000 P f25610539c2a4334b08916bbe4683586 [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:13.513182 12445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:52:13.518450 12445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:52:13.526634 12445 catalog_manager.cc:1347] Generated new cluster ID: 82219849d7e746719b04e3042a44c77e
I20260811 12:52:13.526748 12445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:52:13.549461 12445 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:52:13.551290 12445 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:52:13.560765 12445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f25610539c2a4334b08916bbe4683586: Generated new TSK 0
I20260811 12:52:13.561617 12445 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:52:13.574797 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 12298
W20260811 12:52:13.576237 12410 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:32803
W20260811 12:52:13.576961 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 12298: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:52:13.577113 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 12298
W20260811 12:52:13.609145 12279 connection.cc:535] client connection to 127.7.111.126:41977 recv error: Network error: failed to read from TLS socket (remote: 127.7.111.126:41977): Connection reset by peer (error 104)
2026-08-11T12:52:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2181 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:52:13 dist-test-slave-f0tn krb5kdc[12470](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:52:13 dist-test-slave-f0tn krb5kdc[12470](info): set up 2 sockets
Aug 11 12:52:13 dist-test-slave-f0tn krb5kdc[12470](info): commencing operation
krb5kdc: starting...
W20260811 12:52:14.870180 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.004s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:52:15 dist-test-slave-f0tn krb5kdc[12470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:52:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:52:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:52:15.206130 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:40875
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:35573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:40875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:52:15.552364 12495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:52:15.552640 12495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:52:15.552783 12495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:52:15.559888 12495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:52:15.560001 12495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:52:15.560078 12495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:52:15.560158 12495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:52:15.560215 12495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:52:15.569909 12495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:35573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:40875
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:40875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:52:15.571161 12495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:52:15.572436 12495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:52:15.583683 12511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:15.583760 12507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:15.583737 12508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:52:15.584359 12495 server_base.cc:1034] running on GCE node
I20260811 12:52:15.585103 12495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:52:15.586470 12495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:52:15.587668 12495 hybrid_clock.cc:648] HybridClock initialized: now 1786452735587614 us; error 41 us; skew 500 ppm
I20260811 12:52:15.588004 12495 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:52:15 dist-test-slave-f0tn krb5kdc[12470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452735, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:52:15.591010 12495 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:52:15.592506 12495 webserver.cc:460] Webserver started at http://127.7.111.126:44223/ using document root <none> and password file <none>
I20260811 12:52:15.593212 12495 fs_manager.cc:359] Metadata directory not provided
I20260811 12:52:15.593330 12495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:52:15.593652 12495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:52:15.596652 12495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "5fb58c325fc6447cb746bbcb6d600a25"
format_stamp: "Formatted at 2026-08-11 12:52:15 on dist-test-slave-f0tn"
server_key: "006a7ca818ecef5a9c4b5aa86439edc6"
server_key_iv: "4be874ba0875a9032278aa1b50791e72"
server_key_version: "encryptionkey@0"
I20260811 12:52:15.597445 12495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "5fb58c325fc6447cb746bbcb6d600a25"
format_stamp: "Formatted at 2026-08-11 12:52:15 on dist-test-slave-f0tn"
server_key: "006a7ca818ecef5a9c4b5aa86439edc6"
server_key_iv: "4be874ba0875a9032278aa1b50791e72"
server_key_version: "encryptionkey@0"
I20260811 12:52:15.602591 12495 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 12:52:15.606469 12524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:52:15.607841 12495 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:52:15.608001 12495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "5fb58c325fc6447cb746bbcb6d600a25"
format_stamp: "Formatted at 2026-08-11 12:52:15 on dist-test-slave-f0tn"
server_key: "006a7ca818ecef5a9c4b5aa86439edc6"
server_key_iv: "4be874ba0875a9032278aa1b50791e72"
server_key_version: "encryptionkey@0"
I20260811 12:52:15.608191 12495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:52:15.613967 12495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:52:15.617774 12495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:52:15.618036 12495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:52:15.635341 12495 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:40875
I20260811 12:52:15.635371 12624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:40875 every 8 connection(s)
I20260811 12:52:15.636670 12495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:52:15.639235 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 12495
I20260811 12:52:15.639402 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:52:15.639853 7613 external_mini_cluster.cc:1428] Setting key 2a40568232c6c570b66170824e13c7ec
I20260811 12:52:15.640465 12625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:52:15.647914 12628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:15.641633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33779 (local address 127.7.111.126:40875)
0811 12:52:15.642454 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:15.642469 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:15.642520 (+ 51us) server_negotiation.cc:407] Connection header received
0811 12:52:15.644122 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:15.644150 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:15.644582 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:15.645020 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:15.646006 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:15.646014 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:15.646032 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:52:15.646049 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:15.646140 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:52:15.646166 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:15.646388 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:15.646523 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 12:52:15.646732 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":322,"threads_started":1}
I20260811 12:52:15.651310 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:52:15.653998 12625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fb58c325fc6447cb746bbcb6d600a25: Bootstrap starting.
I20260811 12:52:15.657353 12625 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fb58c325fc6447cb746bbcb6d600a25: Neither blocks nor log segments found. Creating new log.
I20260811 12:52:15.657773 12628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:15.655434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44639 (local address 127.7.111.126:40875)
0811 12:52:15.655589 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:15.655595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:15.655944 (+ 349us) server_negotiation.cc:407] Connection header received
0811 12:52:15.656102 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:15.656105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:15.656148 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:15.656222 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:15.657093 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:15.657099 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:15.657126 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:52:15.657143 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:15.657195 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:52:15.657214 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:15.657321 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:15.657446 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 12:52:15.657504 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 12:52:15.658586 12625 log.cc:824] T 00000000000000000000000000000000 P 5fb58c325fc6447cb746bbcb6d600a25: Log is configured to *not* fsync() on all Append() calls
I20260811 12:52:15.661262 12625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fb58c325fc6447cb746bbcb6d600a25: No bootstrap required, opened a new log
I20260811 12:52:15.668493 12625 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fb58c325fc6447cb746bbcb6d600a25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb58c325fc6447cb746bbcb6d600a25" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40875 } }
I20260811 12:52:15.668861 12625 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5fb58c325fc6447cb746bbcb6d600a25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:52:15.668954 12625 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fb58c325fc6447cb746bbcb6d600a25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fb58c325fc6447cb746bbcb6d600a25, State: Initialized, Role: FOLLOWER
I20260811 12:52:15.669551 12625 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fb58c325fc6447cb746bbcb6d600a25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb58c325fc6447cb746bbcb6d600a25" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40875 } }
I20260811 12:52:15.669831 12625 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5fb58c325fc6447cb746bbcb6d600a25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:52:15.669971 12625 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fb58c325fc6447cb746bbcb6d600a25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:52:15.670157 12625 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fb58c325fc6447cb746bbcb6d600a25 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:52:15.672267 12625 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fb58c325fc6447cb746bbcb6d600a25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb58c325fc6447cb746bbcb6d600a25" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40875 } }
I20260811 12:52:15.672744 12625 leader_election.cc:302] T 00000000000000000000000000000000 P 5fb58c325fc6447cb746bbcb6d600a25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5fb58c325fc6447cb746bbcb6d600a25; no voters:
I20260811 12:52:15.673277 12625 leader_election.cc:288] T 00000000000000000000000000000000 P 5fb58c325fc6447cb746bbcb6d600a25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:52:15.673426 12640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fb58c325fc6447cb746bbcb6d600a25 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:52:15.674011 12640 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fb58c325fc6447cb746bbcb6d600a25 [term 1 LEADER]: Becoming Leader. State: Replica: 5fb58c325fc6447cb746bbcb6d600a25, State: Running, Role: LEADER
I20260811 12:52:15.674595 12640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fb58c325fc6447cb746bbcb6d600a25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb58c325fc6447cb746bbcb6d600a25" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40875 } }
I20260811 12:52:15.675264 12625 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fb58c325fc6447cb746bbcb6d600a25 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:52:15.677433 12642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fb58c325fc6447cb746bbcb6d600a25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5fb58c325fc6447cb746bbcb6d600a25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb58c325fc6447cb746bbcb6d600a25" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40875 } } }
I20260811 12:52:15.677476 12645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fb58c325fc6447cb746bbcb6d600a25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fb58c325fc6447cb746bbcb6d600a25. Latest consensus state: current_term: 1 leader_uuid: "5fb58c325fc6447cb746bbcb6d600a25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb58c325fc6447cb746bbcb6d600a25" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40875 } } }
I20260811 12:52:15.677897 12645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fb58c325fc6447cb746bbcb6d600a25 [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:15.677906 12642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fb58c325fc6447cb746bbcb6d600a25 [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:15.678901 12652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:52:15.684034 12652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:52:15.693325 12652 catalog_manager.cc:1347] Generated new cluster ID: 14358d19ffdd4ff6b7ecbc927dadf149
I20260811 12:52:15.693447 12652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:52:15.715004 12652 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:52:15.715392 12652 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:52:15.730693 12652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5fb58c325fc6447cb746bbcb6d600a25: Generated new TSK 0
I20260811 12:52:15.731433 12652 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:52:15.745846 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 12495
W20260811 12:52:15.746868 12609 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33779
W20260811 12:52:15.747510 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 12495: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:52:15.747630 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 12495
2026-08-11T12:52:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2162 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:52:15 dist-test-slave-f0tn krb5kdc[12681](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:52:15 dist-test-slave-f0tn krb5kdc[12681](info): set up 2 sockets
Aug 11 12:52:15 dist-test-slave-f0tn krb5kdc[12681](info): commencing operation
krb5kdc: starting...
W20260811 12:52:17.010561 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:52:17 dist-test-slave-f0tn krb5kdc[12681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:52:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:52:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:52:17.370780 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:46231
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:44843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:46231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:52:17.716638 12704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:52:17.716951 12704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:52:17.717109 12704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:52:17.724501 12704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:52:17.724620 12704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:52:17.724709 12704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:52:17.724797 12704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:52:17.724861 12704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:52:17.734851 12704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:44843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:46231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:46231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:52:17.736241 12704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:52:17.737674 12704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:52:17.749747 12725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:17.749852 12723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:17.749828 12720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:52:17.750716 12704 server_base.cc:1034] running on GCE node
I20260811 12:52:17.751499 12704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:52:17.752933 12704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:52:17.754163 12704 hybrid_clock.cc:648] HybridClock initialized: now 1786452737754089 us; error 58 us; skew 500 ppm
I20260811 12:52:17.754525 12704 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:52:17 dist-test-slave-f0tn krb5kdc[12681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452737, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:52:17.757602 12704 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:52:17.759073 12704 webserver.cc:460] Webserver started at http://127.7.111.126:39983/ using document root <none> and password file <none>
I20260811 12:52:17.759750 12704 fs_manager.cc:359] Metadata directory not provided
I20260811 12:52:17.759872 12704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:52:17.760172 12704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:52:17.763300 12704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "643e9166962241ff9114fe313253b4f9"
format_stamp: "Formatted at 2026-08-11 12:52:17 on dist-test-slave-f0tn"
server_key: "6c989f640aebec29b85d2b07f6a860a8"
server_key_iv: "7236230bc84a66a505d8589196669795"
server_key_version: "encryptionkey@0"
I20260811 12:52:17.764050 12704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "643e9166962241ff9114fe313253b4f9"
format_stamp: "Formatted at 2026-08-11 12:52:17 on dist-test-slave-f0tn"
server_key: "6c989f640aebec29b85d2b07f6a860a8"
server_key_iv: "7236230bc84a66a505d8589196669795"
server_key_version: "encryptionkey@0"
I20260811 12:52:17.768957 12704 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 12:52:17.772292 12742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:52:17.773524 12704 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:52:17.773679 12704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "643e9166962241ff9114fe313253b4f9"
format_stamp: "Formatted at 2026-08-11 12:52:17 on dist-test-slave-f0tn"
server_key: "6c989f640aebec29b85d2b07f6a860a8"
server_key_iv: "7236230bc84a66a505d8589196669795"
server_key_version: "encryptionkey@0"
I20260811 12:52:17.773845 12704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:52:17.788820 12704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:52:17.792826 12704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:52:17.793076 12704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:52:17.810420 12704 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:46231
I20260811 12:52:17.810427 12820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:46231 every 8 connection(s)
I20260811 12:52:17.811921 12704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:52:17.814332 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 12704
I20260811 12:52:17.814507 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:52:17.815076 7613 external_mini_cluster.cc:1428] Setting key 46b2b54e20c1c6039277012ddc824a82
I20260811 12:52:17.816161 12823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 12:52:17 dist-test-slave-f0tn krb5kdc[12681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:52:17.829085 12823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 643e9166962241ff9114fe313253b4f9: Bootstrap starting.
I20260811 12:52:17.832382 12823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 643e9166962241ff9114fe313253b4f9: Neither blocks nor log segments found. Creating new log.
I20260811 12:52:17.833475 12823 log.cc:824] T 00000000000000000000000000000000 P 643e9166962241ff9114fe313253b4f9: Log is configured to *not* fsync() on all Append() calls
I20260811 12:52:17.836621 12823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 643e9166962241ff9114fe313253b4f9: No bootstrap required, opened a new log
I20260811 12:52:17.837371 12826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:17.816881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54761 (local address 127.7.111.126:46231)
0811 12:52:17.817666 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:17.817684 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:17.817726 (+ 42us) server_negotiation.cc:407] Connection header received
0811 12:52:17.819292 (+ 1566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:17.819312 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:17.819709 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:17.820034 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:17.821794 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:17.823128 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:17.824855 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:17.825878 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:17.826060 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:17.830345 (+ 4285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:17.830352 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:17.830366 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:17.830401 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:17.833336 (+ 2935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:17.834163 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:17.834171 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:17.834180 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:17.834410 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:17.834754 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:17.834760 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:17.834763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:17.835281 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:17.835481 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:17.835979 (+ 498us) server_negotiation.cc:299] Negotiation successful
0811 12:52:17.836293 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":343,"threads_started":1}
I20260811 12:52:17.840905 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:52:17.845685 12823 raft_consensus.cc:348] T 00000000000000000000000000000000 P 643e9166962241ff9114fe313253b4f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643e9166962241ff9114fe313253b4f9" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46231 } }
I20260811 12:52:17.846077 12823 raft_consensus.cc:374] T 00000000000000000000000000000000 P 643e9166962241ff9114fe313253b4f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:52:17.846187 12823 raft_consensus.cc:729] T 00000000000000000000000000000000 P 643e9166962241ff9114fe313253b4f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643e9166962241ff9114fe313253b4f9, State: Initialized, Role: FOLLOWER
I20260811 12:52:17.846858 12823 consensus_queue.cc:260] T 00000000000000000000000000000000 P 643e9166962241ff9114fe313253b4f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643e9166962241ff9114fe313253b4f9" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46231 } }
I20260811 12:52:17.847136 12823 raft_consensus.cc:388] T 00000000000000000000000000000000 P 643e9166962241ff9114fe313253b4f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:52:17.847285 12823 raft_consensus.cc:482] T 00000000000000000000000000000000 P 643e9166962241ff9114fe313253b4f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:52:17.847465 12823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 643e9166962241ff9114fe313253b4f9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:52:17.850353 12823 raft_consensus.cc:504] T 00000000000000000000000000000000 P 643e9166962241ff9114fe313253b4f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643e9166962241ff9114fe313253b4f9" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46231 } }
I20260811 12:52:17.850899 12823 leader_election.cc:302] T 00000000000000000000000000000000 P 643e9166962241ff9114fe313253b4f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 643e9166962241ff9114fe313253b4f9; no voters:
I20260811 12:52:17.851536 12823 leader_election.cc:288] T 00000000000000000000000000000000 P 643e9166962241ff9114fe313253b4f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:52:17.851646 12839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 643e9166962241ff9114fe313253b4f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:52:17.853338 12839 raft_consensus.cc:686] T 00000000000000000000000000000000 P 643e9166962241ff9114fe313253b4f9 [term 1 LEADER]: Becoming Leader. State: Replica: 643e9166962241ff9114fe313253b4f9, State: Running, Role: LEADER
I20260811 12:52:17.853623 12823 sys_catalog.cc:564] T 00000000000000000000000000000000 P 643e9166962241ff9114fe313253b4f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:52:17.853935 12839 consensus_queue.cc:237] T 00000000000000000000000000000000 P 643e9166962241ff9114fe313253b4f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643e9166962241ff9114fe313253b4f9" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46231 } }
I20260811 12:52:17.859051 12840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 643e9166962241ff9114fe313253b4f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "643e9166962241ff9114fe313253b4f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643e9166962241ff9114fe313253b4f9" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46231 } } }
I20260811 12:52:17.859434 12840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 643e9166962241ff9114fe313253b4f9 [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:17.862519 12826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:17.845992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54739 (local address 127.7.111.126:46231)
0811 12:52:17.846161 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:17.846171 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:17.846709 (+ 538us) server_negotiation.cc:407] Connection header received
0811 12:52:17.846994 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:17.846998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:17.847057 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:17.847154 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:17.849009 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:17.849782 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:17.851434 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:17.852118 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:17.852262 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:17.855430 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:17.855437 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:17.855442 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:17.855481 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:17.859357 (+ 3876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:17.860257 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:17.860263 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:17.860267 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:17.860446 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:17.860906 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:17.860911 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:17.860915 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:17.861285 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:17.861612 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:17.861907 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 12:52:17.862136 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:52:17.864650 12842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 643e9166962241ff9114fe313253b4f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 643e9166962241ff9114fe313253b4f9. Latest consensus state: current_term: 1 leader_uuid: "643e9166962241ff9114fe313253b4f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643e9166962241ff9114fe313253b4f9" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 46231 } } }
I20260811 12:52:17.864950 12842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 643e9166962241ff9114fe313253b4f9 [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:17.866416 12853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:52:17.871731 12853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:52:17.880558 12853 catalog_manager.cc:1347] Generated new cluster ID: f7b87928975c4c7f96a64595765090da
I20260811 12:52:17.880651 12853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:52:17.893419 12853 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:52:17.895223 12853 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:52:17.904960 12853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 643e9166962241ff9114fe313253b4f9: Generated new TSK 0
I20260811 12:52:17.906064 12853 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:52:17.920327 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 12704
I20260811 12:52:17.921548 12816 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:52:18.012302 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 12704
2026-08-11T12:52:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2264 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:52:18 dist-test-slave-f0tn krb5kdc[12875](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:52:18 dist-test-slave-f0tn krb5kdc[12875](info): set up 2 sockets
Aug 11 12:52:18 dist-test-slave-f0tn krb5kdc[12875](info): commencing operation
krb5kdc: starting...
W20260811 12:52:19.288460 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.000s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:52:19 dist-test-slave-f0tn krb5kdc[12875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:52:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:52:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:52:19.645895 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:43421
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:43421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:52:19.992146 12900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:52:19.992434 12900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:52:19.992535 12900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:52:19.999799 12900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:52:19.999905 12900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:52:19.999959 12900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:52:20.000027 12900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:52:20.000072 12900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:52:20.010072 12900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:43421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:43421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:52:20.011385 12900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:52:20.012722 12900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:52:20.024277 12907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:20.024384 12906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:20.024780 12910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:52:20.025467 12900 server_base.cc:1034] running on GCE node
I20260811 12:52:20.026163 12900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:52:20.027483 12900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:52:20.028671 12900 hybrid_clock.cc:648] HybridClock initialized: now 1786452740028617 us; error 47 us; skew 500 ppm
I20260811 12:52:20.028995 12900 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:52:20 dist-test-slave-f0tn krb5kdc[12875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452740, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:52:20.036377 12900 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:52:20.037842 12900 webserver.cc:460] Webserver started at http://127.7.111.126:45605/ using document root <none> and password file <none>
I20260811 12:52:20.038518 12900 fs_manager.cc:359] Metadata directory not provided
I20260811 12:52:20.038627 12900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:52:20.038937 12900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:52:20.041945 12900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "3a50238e75e347a88afbbe17705f8b3e"
format_stamp: "Formatted at 2026-08-11 12:52:20 on dist-test-slave-f0tn"
server_key: "b5c0d710cbad20fd7ac43835e8919d71"
server_key_iv: "2d7620eb462155f32d266b426577fe48"
server_key_version: "encryptionkey@0"
I20260811 12:52:20.042676 12900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "3a50238e75e347a88afbbe17705f8b3e"
format_stamp: "Formatted at 2026-08-11 12:52:20 on dist-test-slave-f0tn"
server_key: "b5c0d710cbad20fd7ac43835e8919d71"
server_key_iv: "2d7620eb462155f32d266b426577fe48"
server_key_version: "encryptionkey@0"
I20260811 12:52:20.047570 12900 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:52:20.050947 12925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:52:20.052181 12900 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:52:20.052327 12900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "3a50238e75e347a88afbbe17705f8b3e"
format_stamp: "Formatted at 2026-08-11 12:52:20 on dist-test-slave-f0tn"
server_key: "b5c0d710cbad20fd7ac43835e8919d71"
server_key_iv: "2d7620eb462155f32d266b426577fe48"
server_key_version: "encryptionkey@0"
I20260811 12:52:20.052495 12900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:52:20.074761 12900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:52:20.078608 12900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:52:20.078871 12900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:52:20.095678 12900 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:43421
I20260811 12:52:20.095713 13010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:43421 every 8 connection(s)
I20260811 12:52:20.097025 12900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:52:20.099085 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 12900
I20260811 12:52:20.099277 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:52:20.099845 7613 external_mini_cluster.cc:1428] Setting key 9feafd3ae1870ad750ee121fc2bbb75b
I20260811 12:52:20.101563 13011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:52:20.112985 13014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:20.101719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55565 (local address 127.7.111.126:43421)
0811 12:52:20.102515 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:20.102527 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:20.102568 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:52:20.104201 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:20.104224 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:20.104616 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:20.104967 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:20.105576 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:20.106886 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:20.108667 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:20.109324 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:20.109485 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:20.110721 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:20.110728 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:20.110746 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:52:20.111308 (+ 562us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:20.111383 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:52:20.111411 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:20.111504 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:20.111753 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 12:52:20.112016 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":366,"threads_started":1}
I20260811 12:52:20.114290 13011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a50238e75e347a88afbbe17705f8b3e: Bootstrap starting.
I20260811 12:52:20.116508 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:52:20.117640 13011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a50238e75e347a88afbbe17705f8b3e: Neither blocks nor log segments found. Creating new log.
I20260811 12:52:20.118867 13011 log.cc:824] T 00000000000000000000000000000000 P 3a50238e75e347a88afbbe17705f8b3e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:52:20.122144 13011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a50238e75e347a88afbbe17705f8b3e: No bootstrap required, opened a new log
I20260811 12:52:20.127988 13014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:20.121682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47001 (local address 127.7.111.126:43421)
0811 12:52:20.121879 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:20.121885 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:20.122272 (+ 387us) server_negotiation.cc:407] Connection header received
0811 12:52:20.122443 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:20.122448 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:20.122524 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:20.122644 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:20.123117 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:20.124225 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:20.125493 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:20.126174 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:20.126301 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:20.127291 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:20.127298 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:20.127303 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:52:20.127323 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:20.127373 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:52:20.127391 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:20.127484 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:20.127558 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 12:52:20.127647 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 12:52:20.131021 13011 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a50238e75e347a88afbbe17705f8b3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a50238e75e347a88afbbe17705f8b3e" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 43421 } }
I20260811 12:52:20.131371 13011 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a50238e75e347a88afbbe17705f8b3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:52:20.131448 13011 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a50238e75e347a88afbbe17705f8b3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a50238e75e347a88afbbe17705f8b3e, State: Initialized, Role: FOLLOWER
I20260811 12:52:20.132022 13011 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a50238e75e347a88afbbe17705f8b3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a50238e75e347a88afbbe17705f8b3e" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 43421 } }
I20260811 12:52:20.132272 13011 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a50238e75e347a88afbbe17705f8b3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:52:20.132404 13011 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a50238e75e347a88afbbe17705f8b3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:52:20.132599 13011 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a50238e75e347a88afbbe17705f8b3e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:52:20.134824 13011 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a50238e75e347a88afbbe17705f8b3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a50238e75e347a88afbbe17705f8b3e" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 43421 } }
I20260811 12:52:20.135366 13011 leader_election.cc:302] T 00000000000000000000000000000000 P 3a50238e75e347a88afbbe17705f8b3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a50238e75e347a88afbbe17705f8b3e; no voters:
I20260811 12:52:20.135895 13011 leader_election.cc:288] T 00000000000000000000000000000000 P 3a50238e75e347a88afbbe17705f8b3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:52:20.136001 13027 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a50238e75e347a88afbbe17705f8b3e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:52:20.136646 13027 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a50238e75e347a88afbbe17705f8b3e [term 1 LEADER]: Becoming Leader. State: Replica: 3a50238e75e347a88afbbe17705f8b3e, State: Running, Role: LEADER
I20260811 12:52:20.137213 13027 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a50238e75e347a88afbbe17705f8b3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a50238e75e347a88afbbe17705f8b3e" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 43421 } }
I20260811 12:52:20.137909 13011 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a50238e75e347a88afbbe17705f8b3e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:52:20.140056 13028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a50238e75e347a88afbbe17705f8b3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a50238e75e347a88afbbe17705f8b3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a50238e75e347a88afbbe17705f8b3e" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 43421 } } }
I20260811 12:52:20.140479 13028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a50238e75e347a88afbbe17705f8b3e [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:20.140957 13029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a50238e75e347a88afbbe17705f8b3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a50238e75e347a88afbbe17705f8b3e. Latest consensus state: current_term: 1 leader_uuid: "3a50238e75e347a88afbbe17705f8b3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a50238e75e347a88afbbe17705f8b3e" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 43421 } } }
I20260811 12:52:20.141270 13029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a50238e75e347a88afbbe17705f8b3e [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:20.146502 13037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:52:20.151903 13037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:52:20.161456 13037 catalog_manager.cc:1347] Generated new cluster ID: ce933248a4a64c9f905cc583b77c89ef
I20260811 12:52:20.161598 13037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:52:20.189329 13037 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:52:20.190771 13037 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:52:20.210568 13037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a50238e75e347a88afbbe17705f8b3e: Generated new TSK 0
I20260811 12:52:20.211721 13037 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:52:20.229913 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 12900
W20260811 12:52:20.231154 13002 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55565
W20260811 12:52:20.231930 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 12900: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:52:20.232092 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 12900
2026-08-11T12:52:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2219 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:52:20 dist-test-slave-f0tn krb5kdc[13057](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:52:20 dist-test-slave-f0tn krb5kdc[13057](info): set up 2 sockets
Aug 11 12:52:20 dist-test-slave-f0tn krb5kdc[13057](info): commencing operation
krb5kdc: starting...
W20260811 12:52:21.550030 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.000s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:52:21 dist-test-slave-f0tn krb5kdc[13057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:52:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:52:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:52:21.898574 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:40557
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:43123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:40557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:52:22.240957 13076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:52:22.241261 13076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:52:22.241418 13076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:52:22.248558 13076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:52:22.248672 13076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:52:22.248752 13076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:52:22.248835 13076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:52:22.248896 13076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:52:22.258705 13076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:43123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:40557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:40557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:52:22.260023 13076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:52:22.261423 13076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:52:22.273160 13085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:22.273418 13084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:22.273669 13087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:52:22.274151 13076 server_base.cc:1034] running on GCE node
I20260811 12:52:22.274943 13076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:52:22.276368 13076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:52:22.277580 13076 hybrid_clock.cc:648] HybridClock initialized: now 1786452742277523 us; error 34 us; skew 500 ppm
I20260811 12:52:22.277935 13076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:52:22 dist-test-slave-f0tn krb5kdc[13057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452742, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:52:22.280959 13076 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:52:22.282430 13076 webserver.cc:460] Webserver started at http://127.7.111.126:36497/ using document root <none> and password file <none>
I20260811 12:52:22.283109 13076 fs_manager.cc:359] Metadata directory not provided
I20260811 12:52:22.283232 13076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:52:22.283531 13076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:52:22.286633 13076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "1489862ab49a405e990463f255e075f8"
format_stamp: "Formatted at 2026-08-11 12:52:22 on dist-test-slave-f0tn"
server_key: "8122985b53c102886ffcdb124d84aaab"
server_key_iv: "c970279b2d7910ae0e8c2f0be1e620fa"
server_key_version: "encryptionkey@0"
I20260811 12:52:22.287405 13076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "1489862ab49a405e990463f255e075f8"
format_stamp: "Formatted at 2026-08-11 12:52:22 on dist-test-slave-f0tn"
server_key: "8122985b53c102886ffcdb124d84aaab"
server_key_iv: "c970279b2d7910ae0e8c2f0be1e620fa"
server_key_version: "encryptionkey@0"
I20260811 12:52:22.292539 13076 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 12:52:22.295991 13095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:52:22.297271 13076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:52:22.297472 13076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "1489862ab49a405e990463f255e075f8"
format_stamp: "Formatted at 2026-08-11 12:52:22 on dist-test-slave-f0tn"
server_key: "8122985b53c102886ffcdb124d84aaab"
server_key_iv: "c970279b2d7910ae0e8c2f0be1e620fa"
server_key_version: "encryptionkey@0"
I20260811 12:52:22.297655 13076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:52:22.303256 13076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:52:22.307227 13076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:52:22.307503 13076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:52:22.324666 13076 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:40557
I20260811 12:52:22.324684 13199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:40557 every 8 connection(s)
I20260811 12:52:22.326148 13076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:52:22.330013 13200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:52:22.331341 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 13076
I20260811 12:52:22.331532 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:52:22.332099 7613 external_mini_cluster.cc:1428] Setting key ab08b27179eb28a245d6f13867ae8081
I20260811 12:52:22.339751 13207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:22.333919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40507 (local address 127.7.111.126:40557)
0811 12:52:22.334764 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:22.334776 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:22.334822 (+ 46us) server_negotiation.cc:407] Connection header received
0811 12:52:22.336445 (+ 1623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:22.336469 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:22.336826 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:22.337171 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:22.338227 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:22.338234 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:22.338247 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:52:22.338262 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:22.338320 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:52:22.338343 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:22.338507 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:22.338603 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 12:52:22.338774 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":368,"threads_started":1}
I20260811 12:52:22.343662 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:52:22.343940 13200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1489862ab49a405e990463f255e075f8: Bootstrap starting.
I20260811 12:52:22.347106 13200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1489862ab49a405e990463f255e075f8: Neither blocks nor log segments found. Creating new log.
I20260811 12:52:22.348244 13200 log.cc:824] T 00000000000000000000000000000000 P 1489862ab49a405e990463f255e075f8: Log is configured to *not* fsync() on all Append() calls
I20260811 12:52:22.350450 13207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:22.347980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55041 (local address 127.7.111.126:40557)
0811 12:52:22.348168 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:22.348174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:22.348567 (+ 393us) server_negotiation.cc:407] Connection header received
0811 12:52:22.348720 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:22.348727 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:22.348794 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:22.348900 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:22.349844 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:22.349850 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:22.349855 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:52:22.349873 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:22.349910 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:52:22.349924 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:22.350019 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:22.350129 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 12:52:22.350189 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:52:22.351250 13200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1489862ab49a405e990463f255e075f8: No bootstrap required, opened a new log
I20260811 12:52:22.358922 13200 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1489862ab49a405e990463f255e075f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1489862ab49a405e990463f255e075f8" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40557 } }
I20260811 12:52:22.359292 13200 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1489862ab49a405e990463f255e075f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:52:22.359380 13200 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1489862ab49a405e990463f255e075f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1489862ab49a405e990463f255e075f8, State: Initialized, Role: FOLLOWER
I20260811 12:52:22.359939 13200 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1489862ab49a405e990463f255e075f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1489862ab49a405e990463f255e075f8" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40557 } }
I20260811 12:52:22.360159 13200 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1489862ab49a405e990463f255e075f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:52:22.360291 13200 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1489862ab49a405e990463f255e075f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:52:22.360437 13200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1489862ab49a405e990463f255e075f8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:52:22.362644 13200 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1489862ab49a405e990463f255e075f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1489862ab49a405e990463f255e075f8" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40557 } }
I20260811 12:52:22.363140 13200 leader_election.cc:302] T 00000000000000000000000000000000 P 1489862ab49a405e990463f255e075f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1489862ab49a405e990463f255e075f8; no voters:
I20260811 12:52:22.363865 13200 leader_election.cc:288] T 00000000000000000000000000000000 P 1489862ab49a405e990463f255e075f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:52:22.364044 13216 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1489862ab49a405e990463f255e075f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:52:22.364682 13216 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1489862ab49a405e990463f255e075f8 [term 1 LEADER]: Becoming Leader. State: Replica: 1489862ab49a405e990463f255e075f8, State: Running, Role: LEADER
I20260811 12:52:22.365298 13216 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1489862ab49a405e990463f255e075f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1489862ab49a405e990463f255e075f8" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40557 } }
I20260811 12:52:22.365887 13200 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1489862ab49a405e990463f255e075f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:52:22.368290 13218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1489862ab49a405e990463f255e075f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1489862ab49a405e990463f255e075f8. Latest consensus state: current_term: 1 leader_uuid: "1489862ab49a405e990463f255e075f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1489862ab49a405e990463f255e075f8" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40557 } } }
I20260811 12:52:22.368355 13217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1489862ab49a405e990463f255e075f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1489862ab49a405e990463f255e075f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1489862ab49a405e990463f255e075f8" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40557 } } }
I20260811 12:52:22.368749 13217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1489862ab49a405e990463f255e075f8 [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:22.368749 13218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1489862ab49a405e990463f255e075f8 [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:22.369675 13232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:52:22.375164 13232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:52:22.383463 13232 catalog_manager.cc:1347] Generated new cluster ID: 28f5453c4390483e8e5ece732ba6a054
I20260811 12:52:22.383597 13232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:52:22.438450 13232 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:52:22.438911 13232 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:52:22.444758 13232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1489862ab49a405e990463f255e075f8: Generated new TSK 0
I20260811 12:52:22.445631 13232 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:52:22.460925 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 13076
W20260811 12:52:22.462060 13191 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40507
W20260811 12:52:22.462782 7613 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 13076: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:52:22.462929 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 13076
2026-08-11T12:52:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2234 ms)
[----------] 18 tests from AuthTokenIssuingTest (39740 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:52:22 dist-test-slave-f0tn krb5kdc[13251](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:52:22 dist-test-slave-f0tn krb5kdc[13251](info): set up 2 sockets
Aug 11 12:52:22 dist-test-slave-f0tn krb5kdc[13251](info): commencing operation
krb5kdc: starting...
W20260811 12:52:23.738771 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.000s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:52:23 dist-test-slave-f0tn krb5kdc[13251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:52:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:52:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:52:24.074698 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:45777
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:37549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:45777,127.7.111.125:39411,127.7.111.124:46261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:52:24.432065 13282 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:52:24.432425 13282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:52:24.432528 13282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:52:24.432649 13282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:52:24.439919 13282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:52:24.440017 13282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:52:24.440102 13282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:52:24.440167 13282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:52:24.440254 13282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:52:24.450193 13282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:37549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:45777,127.7.111.125:39411,127.7.111.124:46261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:45777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:52:24.451514 13282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:52:24.453029 13282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:52:24.464733 13288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:24.464677 13289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:24.464708 13292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:52:24.465863 13282 server_base.cc:1034] running on GCE node
I20260811 12:52:24.466603 13282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:52:24.467926 13282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:52:24.469121 13282 hybrid_clock.cc:648] HybridClock initialized: now 1786452744469054 us; error 43 us; skew 500 ppm
I20260811 12:52:24.469460 13282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:52:24 dist-test-slave-f0tn krb5kdc[13251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452744, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:52:24.472407 13282 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:52:24.473816 13282 webserver.cc:460] Webserver started at http://127.7.111.126:40581/ using document root <none> and password file <none>
I20260811 12:52:24.474438 13282 fs_manager.cc:359] Metadata directory not provided
I20260811 12:52:24.474522 13282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:52:24.474797 13282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:52:24.477725 13282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "4bc1b5df7ed14c0ab09ec51328743c5c"
format_stamp: "Formatted at 2026-08-11 12:52:24 on dist-test-slave-f0tn"
server_key: "1179a34916be294e0e3fea1f46ce338c"
server_key_iv: "20a1f04e53e011e5862d98f237c69bad"
server_key_version: "encryptionkey@0"
I20260811 12:52:24.478533 13282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "4bc1b5df7ed14c0ab09ec51328743c5c"
format_stamp: "Formatted at 2026-08-11 12:52:24 on dist-test-slave-f0tn"
server_key: "1179a34916be294e0e3fea1f46ce338c"
server_key_iv: "20a1f04e53e011e5862d98f237c69bad"
server_key_version: "encryptionkey@0"
I20260811 12:52:24.483634 13282 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:52:24.487233 13303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:52:24.488636 13282 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:52:24.488808 13282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "4bc1b5df7ed14c0ab09ec51328743c5c"
format_stamp: "Formatted at 2026-08-11 12:52:24 on dist-test-slave-f0tn"
server_key: "1179a34916be294e0e3fea1f46ce338c"
server_key_iv: "20a1f04e53e011e5862d98f237c69bad"
server_key_version: "encryptionkey@0"
I20260811 12:52:24.489022 13282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:52:24.517524 13282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:52:24.522153 13282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:52:24.522416 13282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:52:24.539732 13282 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:45777
I20260811 12:52:24.539752 13401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:45777 every 8 connection(s)
I20260811 12:52:24.541148 13282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:52:24.544871 13402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:52:24.548671 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 13282
I20260811 12:52:24.548916 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:52:24.549774 7613 external_mini_cluster.cc:1428] Setting key 3b5389633c9403642415c0356ce419a6
I20260811 12:52:24.552922 13402 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.126" port: 45777 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:52:24 dist-test-slave-f0tn krb5kdc[13251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452744, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:52:24.575906 13409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:24.556757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42252 (local address 127.7.111.126:45777)
0811 12:52:24.558495 (+ 1738us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:24.558504 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:24.558573 (+ 69us) server_negotiation.cc:407] Connection header received
0811 12:52:24.560085 (+ 1512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:24.560091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:24.560342 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:24.560531 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:24.561931 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:24.563142 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:24.564526 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:24.565070 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:24.565253 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:24.568809 (+ 3556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:24.568818 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:24.568831 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:24.568862 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:24.571613 (+ 2751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:24.572466 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:24.572473 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:24.572475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:24.572649 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:24.573084 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:24.573088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:24.573090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:24.573669 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:24.573835 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:24.574544 (+ 709us) server_negotiation.cc:299] Negotiation successful
0811 12:52:24.574810 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1518,"thread_start_us":1291,"threads_started":1}
I20260811 12:52:24.575932 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:24.556756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:45777 (local address 127.0.0.1:42252)
0811 12:52:24.557650 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:24.557713 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 12:52:24.559722 (+ 2009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:52:24.560575 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:52:24.560582 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:52:24.561303 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:52:24.561704 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:24.561725 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:24.563300 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:24.563303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:24.564375 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:24.564387 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:24.565304 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:24.565311 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:24.565515 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:24.566174 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:52:24.566197 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:52:24.568567 (+ 2370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:52:24.571758 (+ 3191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:24.571771 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:24.571784 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:24.572324 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:24.572767 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:24.572772 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:24.572775 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:24.572976 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:24.573825 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:52:24.573836 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:52:24.574142 (+ 306us) client_negotiation.cc:769] Sending connection context
0811 12:52:24.574401 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 12:52:24.575010 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"mutex_wait_us":109,"thread_start_us":358,"threads_started":1}
I20260811 12:52:24.578966 13402 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:52:24.580122 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:24.579654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:39411 (local address 127.0.0.1:53414)
0811 12:52:24.579831 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:24.579963 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 12:52:24.580422 13307 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.111.125:39411: connect: Connection refused (error 111)
I20260811 12:52:24.581993 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:24.581574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:39411 (local address 127.0.0.1:53420)
0811 12:52:24.581765 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:24.581847 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 12:52:24.582244 13402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.125:39411: Network error: Client connection negotiation failed: client connection to 127.7.111.125:39411: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.7.111.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.125.keytab.
Entry for principal kudu/127.7.111.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.125.keytab.
I20260811 12:52:24.628170 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.125:39411
--webserver_interface=127.7.111.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.125.keytab
--principal=kudu/127.7.111.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:37549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:45777,127.7.111.125:39411,127.7.111.124:46261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:52:24.631634 13402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } attempt: 1
I20260811 12:52:24.632965 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:24.632453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:39411 (local address 127.0.0.1:53434)
0811 12:52:24.632689 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:24.632826 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 12:52:24.634102 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:24.633715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:39411 (local address 127.0.0.1:53442)
0811 12:52:24.633886 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:24.633958 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 12:52:24.634315 13402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.125:39411: Network error: Client connection negotiation failed: client connection to 127.7.111.125:39411: connect: Connection refused (error 111)
I20260811 12:52:24.702587 13402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } attempt: 2
I20260811 12:52:24.703941 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:24.703353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:39411 (local address 127.0.0.1:39156)
0811 12:52:24.703631 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:24.703762 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 12:52:24.705021 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:24.704604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:39411 (local address 127.0.0.1:39170)
0811 12:52:24.704769 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:24.704869 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 12:52:24.705246 13402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.125:39411: Network error: Client connection negotiation failed: client connection to 127.7.111.125:39411: connect: Connection refused (error 111)
I20260811 12:52:24.796511 13402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } attempt: 3
I20260811 12:52:24.797776 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:24.797232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:39411 (local address 127.0.0.1:39186)
0811 12:52:24.797483 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:24.797599 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 12:52:24.798832 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:24.798401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:39411 (local address 127.0.0.1:39192)
0811 12:52:24.798562 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:24.798648 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 12:52:24.799032 13402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.125:39411: Network error: Client connection negotiation failed: client connection to 127.7.111.125:39411: connect: Connection refused (error 111)
I20260811 12:52:24.942305 13402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } attempt: 4
I20260811 12:52:24.943686 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:24.942996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:39411 (local address 127.0.0.1:39202)
0811 12:52:24.943381 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:24.943500 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191}
I20260811 12:52:24.944911 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:24.944482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:39411 (local address 127.0.0.1:39206)
0811 12:52:24.944651 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:24.944761 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 12:52:24.945164 13402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.125:39411: Network error: Client connection negotiation failed: client connection to 127.7.111.125:39411: connect: Connection refused (error 111)
W20260811 12:52:24.975455 13414 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:52:24.975790 13414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:52:24.975865 13414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:52:24.975945 13414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:52:24.983356 13414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:52:24.983457 13414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:52:24.983520 13414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:52:24.983572 13414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:52:24.983632 13414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:52:24.993671 13414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:37549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:45777,127.7.111.125:39411,127.7.111.124:46261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.125.keytab
--principal=kudu/127.7.111.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.125:39411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.111.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:52:24.994958 13414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:52:24.996366 13414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:52:25.007915 13423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:25.007952 13426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:25.007998 13424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:52:25.008967 13414 server_base.cc:1034] running on GCE node
I20260811 12:52:25.009825 13414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:52:25.011240 13414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:52:25.012473 13414 hybrid_clock.cc:648] HybridClock initialized: now 1786452745012395 us; error 38 us; skew 500 ppm
I20260811 12:52:25.012811 13414 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:52:25 dist-test-slave-f0tn krb5kdc[13251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452745, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:52:25.015996 13414 init.cc:377] Logged in from keytab as kudu/127.7.111.125@KRBTEST.COM (short username kudu)
I20260811 12:52:25.017517 13414 webserver.cc:460] Webserver started at http://127.7.111.125:32847/ using document root <none> and password file <none>
I20260811 12:52:25.018210 13414 fs_manager.cc:359] Metadata directory not provided
I20260811 12:52:25.018321 13414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:52:25.018630 13414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:52:25.021656 13414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-1/data/instance:
uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d"
format_stamp: "Formatted at 2026-08-11 12:52:25 on dist-test-slave-f0tn"
server_key: "919eb6ce1761674127f5b80b4411de95"
server_key_iv: "1c4294c301ab2c40a9cd09f96d38c74a"
server_key_version: "encryptionkey@0"
I20260811 12:52:25.022447 13414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-1/wal/instance:
uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d"
format_stamp: "Formatted at 2026-08-11 12:52:25 on dist-test-slave-f0tn"
server_key: "919eb6ce1761674127f5b80b4411de95"
server_key_iv: "1c4294c301ab2c40a9cd09f96d38c74a"
server_key_version: "encryptionkey@0"
I20260811 12:52:25.027454 13414 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:52:25.030838 13438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:52:25.032119 13414 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:52:25.032284 13414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-1/wal
uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d"
format_stamp: "Formatted at 2026-08-11 12:52:25 on dist-test-slave-f0tn"
server_key: "919eb6ce1761674127f5b80b4411de95"
server_key_iv: "1c4294c301ab2c40a9cd09f96d38c74a"
server_key_version: "encryptionkey@0"
I20260811 12:52:25.032480 13414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:52:25.047513 13414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:52:25.051662 13414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:52:25.051919 13414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:52:25.069360 13414 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.125:39411
I20260811 12:52:25.069394 13534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.125:39411 every 8 connection(s)
I20260811 12:52:25.070721 13414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-1/data/info.pb
I20260811 12:52:25.073798 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 13414
I20260811 12:52:25.073997 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-1/wal/instance
I20260811 12:52:25.074569 7613 external_mini_cluster.cc:1428] Setting key bbb49ce43d4b4d6b0ddf92216e3bf4bf
I20260811 12:52:25.074740 13535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:52:25.083184 13535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.126" port: 45777 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:52:25 dist-test-slave-f0tn krb5kdc[13251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452745, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.125@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:52:25.104318 13537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.086212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42936 (local address 127.7.111.126:45777)
0811 12:52:25.086759 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:25.086766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:25.087718 (+ 952us) server_negotiation.cc:407] Connection header received
0811 12:52:25.090087 (+ 2369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:25.090093 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:25.090164 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:25.090251 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:25.092002 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.092792 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.094408 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.094825 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.094944 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:25.098438 (+ 3494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:25.098444 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:25.098447 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:25.098494 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:25.101094 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.101907 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.101911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.101913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.102036 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.102457 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.102460 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.102462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.102771 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:25.102914 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:25.103874 (+ 960us) server_negotiation.cc:299] Negotiation successful
0811 12:52:25.104049 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":249,"threads_started":1}
I20260811 12:52:25.105744 13538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.086747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:45777 (local address 127.0.0.1:42936)
0811 12:52:25.087529 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.087576 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 12:52:25.089768 (+ 2192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:52:25.090508 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:52:25.090518 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:52:25.091208 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:52:25.091783 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:25.091800 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.092948 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.092952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:25.094258 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:25.094271 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.095052 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.095060 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:25.095286 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:25.095917 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:52:25.095940 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:52:25.098214 (+ 2274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:52:25.101253 (+ 3039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:25.101262 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:25.101277 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:25.101780 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:25.102144 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:25.102149 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:25.102153 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:25.102340 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:25.102933 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:52:25.102941 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:52:25.103407 (+ 466us) client_negotiation.cc:769] Sending connection context
0811 12:52:25.103783 (+ 376us) client_negotiation.cc:240] Negotiation successful
0811 12:52:25.104303 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":288,"threads_started":1}
I20260811 12:52:25.107577 13535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.111.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.124@KRBTEST.COM" created.
Aug 11 12:52:25 dist-test-slave-f0tn krb5kdc[13251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452745, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.125@KRBTEST.COM for kudu/127.7.111.125@KRBTEST.COM
I20260811 12:52:25.122323 13538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.108457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:39411 (local address 127.0.0.1:39222)
0811 12:52:25.108670 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.108689 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:52:25.108797 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:52:25.109897 (+ 1100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:52:25.109901 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:52:25.110276 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:52:25.110440 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:25.110451 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.111898 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.111903 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:25.112995 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:25.113010 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.113890 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.113896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:25.114077 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:25.114657 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:52:25.114679 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:52:25.116826 (+ 2147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:52:25.119839 (+ 3013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:25.119845 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:25.119849 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:25.120224 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:25.120600 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:25.120605 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:25.120608 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:25.120748 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:25.121334 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:52:25.121339 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:52:25.121497 (+ 158us) client_negotiation.cc:769] Sending connection context
0811 12:52:25.121615 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 12:52:25.121907 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 12:52:25.122402 13541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.108579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39222 (local address 127.7.111.125:39411)
0811 12:52:25.109341 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:25.109348 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:25.109413 (+ 65us) server_negotiation.cc:407] Connection header received
0811 12:52:25.109479 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:25.109488 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:25.109747 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:25.109888 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:25.110611 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.111769 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.113142 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.113746 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.113844 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:25.117024 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:25.117037 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:25.117052 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:25.117081 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:25.119716 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.120351 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.120358 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.120360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.120475 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.120848 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.120852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.120854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.121198 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:25.121413 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:25.121800 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 12:52:25.122023 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":431,"threads_started":1}
I20260811 12:52:25.124958 13535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:52:25.126075 13538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.125634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46261 (local address 127.0.0.1:40626)
0811 12:52:25.125808 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.125923 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 12:52:25.126303 13442 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
I20260811 12:52:25.127825 13538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.127506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46261 (local address 127.0.0.1:40634)
0811 12:52:25.127642 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.127721 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 12:52:25.128018 13535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:46261: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.124.keytab.
Entry for principal kudu/127.7.111.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.124.keytab.
I20260811 12:52:25.151973 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.124:46261
--webserver_interface=127.7.111.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.124.keytab
--principal=kudu/127.7.111.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:37549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:45777,127.7.111.125:39411,127.7.111.124:46261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:52:25.177325 13535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } attempt: 1
I20260811 12:52:25.178498 13538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.178024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46261 (local address 127.0.0.1:40644)
0811 12:52:25.178233 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.178341 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 12:52:25.179572 13538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.179249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46261 (local address 127.0.0.1:40660)
0811 12:52:25.179386 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.179464 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 12:52:25.179818 13535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:46261: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
I20260811 12:52:25.244410 13402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } attempt: 5
I20260811 12:52:25.248042 13535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } attempt: 2
I20260811 12:52:25.249120 13538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.248607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46261 (local address 127.0.0.1:40666)
0811 12:52:25.248819 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.248920 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 12:52:25.250346 13538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.249974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46261 (local address 127.0.0.1:40670)
0811 12:52:25.250128 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.250199 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 12:52:25.250600 13535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:46261: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
Aug 11 12:52:25 dist-test-slave-f0tn krb5kdc[13251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452744, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for kudu/127.7.111.125@KRBTEST.COM
I20260811 12:52:25.258049 13541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.245245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39234 (local address 127.7.111.125:39411)
0811 12:52:25.245441 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:25.245446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:25.245516 (+ 70us) server_negotiation.cc:407] Connection header received
0811 12:52:25.245710 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:25.245716 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:25.245796 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:25.245884 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:25.246695 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.247392 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.248679 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.249347 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.249480 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:25.252976 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:25.252981 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:25.252984 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:25.253017 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:25.255552 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.256339 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.256342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.256345 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.256456 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.256856 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.256859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.256861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.257119 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:25.257239 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:25.257678 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 12:52:25.257843 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:52:25.258278 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.245173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:39411 (local address 127.0.0.1:39234)
0811 12:52:25.245416 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.245438 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:52:25.245585 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:52:25.245941 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:52:25.245946 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:52:25.246354 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:52:25.246534 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:25.246544 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.247524 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.247530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:25.248535 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:25.248548 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.249522 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.249528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:25.249756 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:25.250433 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:52:25.250458 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:52:25.252788 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:52:25.255699 (+ 2911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:25.255704 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:25.255708 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:25.256238 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:25.256559 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:25.256564 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:25.256568 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:25.256762 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:25.257230 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:52:25.257235 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:52:25.257445 (+ 210us) client_negotiation.cc:769] Sending connection context
0811 12:52:25.257602 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 12:52:25.257929 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 12:52:25.259187 13402 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:52:25.260335 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.259868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46261 (local address 127.0.0.1:40684)
0811 12:52:25.260057 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.260156 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 12:52:25.261278 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.260980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46261 (local address 127.0.0.1:40688)
0811 12:52:25.261106 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.261170 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 12:52:25.261488 13402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:46261: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
I20260811 12:52:25.312700 13402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } attempt: 1
I20260811 12:52:25.313926 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.313461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46261 (local address 127.0.0.1:40698)
0811 12:52:25.313655 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.313753 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 12:52:25.314960 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.314609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46261 (local address 127.0.0.1:40708)
0811 12:52:25.314767 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.314845 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 12:52:25.315171 13402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:46261: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
I20260811 12:52:25.341864 13535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } attempt: 3
I20260811 12:52:25.343060 13538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.342571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46261 (local address 127.0.0.1:40720)
0811 12:52:25.342799 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.342895 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 12:52:25.344249 13538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.343793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46261 (local address 127.0.0.1:40736)
0811 12:52:25.344038 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.344126 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
W20260811 12:52:25.344465 13535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:46261: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
I20260811 12:52:25.383371 13402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } attempt: 2
I20260811 12:52:25.384508 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.384011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46261 (local address 127.0.0.1:40752)
0811 12:52:25.384241 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.384342 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 12:52:25.385514 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.385133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46261 (local address 127.0.0.1:40756)
0811 12:52:25.385292 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.385362 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 12:52:25.385725 13402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:46261: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
I20260811 12:52:25.487663 13535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } attempt: 4
I20260811 12:52:25.488867 13538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.488386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46261 (local address 127.0.0.1:40762)
0811 12:52:25.488593 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.488687 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 12:52:25.489835 13538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.489507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46261 (local address 127.0.0.1:40778)
0811 12:52:25.489650 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.489728 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 12:52:25.490041 13535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:46261: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
I20260811 12:52:25.491917 13402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } attempt: 3
I20260811 12:52:25.492820 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.492429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46261 (local address 127.0.0.1:40794)
0811 12:52:25.492585 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.492667 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 12:52:25.493839 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.493498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46261 (local address 127.0.0.1:40810)
0811 12:52:25.493675 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.493745 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260811 12:52:25.494059 13402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:46261: Network error: Client connection negotiation failed: client connection to 127.7.111.124:46261: connect: Connection refused (error 111)
W20260811 12:52:25.510573 13547 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:52:25.510897 13547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:52:25.510969 13547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:52:25.511053 13547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:52:25.518323 13547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:52:25.518415 13547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:52:25.518478 13547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:52:25.518528 13547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:52:25.518589 13547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:52:25.528652 13547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:37549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:45777,127.7.111.125:39411,127.7.111.124:46261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.124.keytab
--principal=kudu/127.7.111.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.124:46261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.111.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:52:25.529994 13547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:52:25.531487 13547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:52:25.543531 13563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:25.543721 13561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:25.544060 13567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:52:25.544745 13547 server_base.cc:1034] running on GCE node
I20260811 12:52:25.545593 13547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:52:25.546983 13547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:52:25.548216 13547 hybrid_clock.cc:648] HybridClock initialized: now 1786452745548121 us; error 60 us; skew 500 ppm
I20260811 12:52:25.548547 13547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:52:25 dist-test-slave-f0tn krb5kdc[13251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452745, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:52:25.551590 13547 init.cc:377] Logged in from keytab as kudu/127.7.111.124@KRBTEST.COM (short username kudu)
I20260811 12:52:25.553052 13547 webserver.cc:460] Webserver started at http://127.7.111.124:38525/ using document root <none> and password file <none>
I20260811 12:52:25.553830 13547 fs_manager.cc:359] Metadata directory not provided
I20260811 12:52:25.553943 13547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:52:25.554307 13547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:52:25.557503 13547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-2/data/instance:
uuid: "dd7467a76edf4c288a080e558cdb8cac"
format_stamp: "Formatted at 2026-08-11 12:52:25 on dist-test-slave-f0tn"
server_key: "2389c4b9e1d58473d660f21ab0375ea3"
server_key_iv: "8dbebca4d3453685721cffb784e39f8b"
server_key_version: "encryptionkey@0"
I20260811 12:52:25.558293 13547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-2/wal/instance:
uuid: "dd7467a76edf4c288a080e558cdb8cac"
format_stamp: "Formatted at 2026-08-11 12:52:25 on dist-test-slave-f0tn"
server_key: "2389c4b9e1d58473d660f21ab0375ea3"
server_key_iv: "8dbebca4d3453685721cffb784e39f8b"
server_key_version: "encryptionkey@0"
I20260811 12:52:25.563827 13547 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260811 12:52:25.567734 13580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:52:25.569239 13547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 12:52:25.569442 13547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-2/wal
uuid: "dd7467a76edf4c288a080e558cdb8cac"
format_stamp: "Formatted at 2026-08-11 12:52:25 on dist-test-slave-f0tn"
server_key: "2389c4b9e1d58473d660f21ab0375ea3"
server_key_iv: "8dbebca4d3453685721cffb784e39f8b"
server_key_version: "encryptionkey@0"
I20260811 12:52:25.569674 13547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:52:25.596948 13547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:52:25.601217 13547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:52:25.601519 13547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:52:25.619612 13547 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.124:46261
I20260811 12:52:25.619633 13673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.124:46261 every 8 connection(s)
I20260811 12:52:25.621003 13547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-2/data/info.pb
I20260811 12:52:25.624814 13674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:52:25.626883 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 13547
I20260811 12:52:25.627076 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786452375851340-7613-0/minicluster-data/master-2/wal/instance
I20260811 12:52:25.627645 7613 external_mini_cluster.cc:1428] Setting key 09a3ee93cbffae59fc4ad8309a1d7489
I20260811 12:52:25.632720 13674 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.126" port: 45777 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:52:25 dist-test-slave-f0tn krb5kdc[13251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.124@KRBTEST.COM
Aug 11 12:52:25 dist-test-slave-f0tn krb5kdc[13251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452745, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.124@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:52:25.650405 13678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.635810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42940 (local address 127.7.111.126:45777)
0811 12:52:25.636358 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:25.636365 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:25.636895 (+ 530us) server_negotiation.cc:407] Connection header received
0811 12:52:25.637154 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:25.637158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:25.637230 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:25.637330 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:25.638808 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.639822 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.641199 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.641674 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.641817 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:25.645076 (+ 3259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:25.645082 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:25.645085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:25.645118 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:25.647651 (+ 2533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.648322 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.648328 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.648331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.648458 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.648815 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.648819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.648821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.649069 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:25.649217 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:25.649776 (+ 559us) server_negotiation.cc:299] Negotiation successful
0811 12:52:25.649988 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":226,"threads_started":1}
I20260811 12:52:25.650990 13677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.629598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40812 (local address 127.7.111.124:46261)
0811 12:52:25.630408 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:25.630420 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:25.630467 (+ 47us) server_negotiation.cc:407] Connection header received
0811 12:52:25.632159 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:25.632190 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:25.632575 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:25.632902 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:25.634789 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.636629 (+ 1840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.638417 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.639308 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.639487 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:25.643774 (+ 4287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:25.643783 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:25.643795 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:25.643826 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:25.646727 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.647384 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.647391 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.647399 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.647547 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.647912 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.647915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.647917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.648460 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:25.648772 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:25.649283 (+ 511us) server_negotiation.cc:299] Negotiation successful
0811 12:52:25.649749 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":346,"threads_started":1}
I20260811 12:52:25.651037 13679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.635937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:45777 (local address 127.0.0.1:42940)
0811 12:52:25.636737 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.636786 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 12:52:25.636974 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:52:25.637462 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:52:25.637470 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:52:25.638325 (+ 855us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:52:25.638651 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:25.638661 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.640007 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.640015 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:25.641049 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:25.641062 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.641917 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.641924 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:25.642070 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:25.642676 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:52:25.642701 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:52:25.644878 (+ 2177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:52:25.647775 (+ 2897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:25.647787 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:25.647795 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:25.648206 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:25.648566 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:25.648571 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:25.648575 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:25.648712 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:25.649206 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:52:25.649215 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:52:25.649434 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 12:52:25.649635 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 12:52:25.649907 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":384,"threads_started":1}
I20260811 12:52:25.652616 13674 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:52:25 dist-test-slave-f0tn krb5kdc[13251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452745, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.124@KRBTEST.COM for kudu/127.7.111.125@KRBTEST.COM
Aug 11 12:52:25 dist-test-slave-f0tn krb5kdc[13251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:52:25.665863 13679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.653511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:39411 (local address 127.0.0.1:39244)
0811 12:52:25.653655 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.653674 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:52:25.653790 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:52:25.654275 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:52:25.654280 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:52:25.654732 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:52:25.654870 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:25.654879 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.655840 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.655846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:25.656746 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:25.656759 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.657479 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.657484 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:25.657673 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:25.658190 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:52:25.658214 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:52:25.660176 (+ 1962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:52:25.663511 (+ 3335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:25.663516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:25.663521 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:25.663884 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:25.664277 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:25.664282 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:25.664286 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:25.664423 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:25.664920 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:52:25.664925 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:52:25.665076 (+ 151us) client_negotiation.cc:769] Sending connection context
0811 12:52:25.665193 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 12:52:25.665509 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 12:52:25.665902 13541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.653462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39244 (local address 127.7.111.125:39411)
0811 12:52:25.653633 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:25.653638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:25.653749 (+ 111us) server_negotiation.cc:407] Connection header received
0811 12:52:25.653905 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:25.653910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:25.653988 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:25.654079 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:25.655021 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.655697 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.656894 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.657327 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.657473 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:25.660361 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:25.660367 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:25.660370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:25.660403 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:25.663390 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.663995 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.664000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.664004 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.664163 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.664523 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.664528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.664532 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.664798 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:25.664938 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:25.665302 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 12:52:25.665550 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 12:52:25.666920 13674 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:52:25.668102 13678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.654077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42952 (local address 127.7.111.126:45777)
0811 12:52:25.654251 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:25.654256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:25.654271 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:52:25.654339 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:25.654345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:25.654425 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:25.654523 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:25.655782 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.656479 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.657824 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.658221 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.658331 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:25.662538 (+ 4207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:25.662544 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:25.662547 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:25.662578 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:25.665089 (+ 2511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.665829 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.665835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.665838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.666181 (+ 343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.666608 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.666614 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.666618 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.666979 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:25.667246 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:25.667573 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 12:52:25.667808 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:52:25.671263 13402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } attempt: 4
Aug 11 12:52:25 dist-test-slave-f0tn krb5kdc[13251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452745, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.124@KRBTEST.COM for kudu/127.7.111.124@KRBTEST.COM
Aug 11 12:52:25 dist-test-slave-f0tn krb5kdc[13251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.125@KRBTEST.COM
Aug 11 12:52:25 dist-test-slave-f0tn krb5kdc[13251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452744, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for kudu/127.7.111.124@KRBTEST.COM
I20260811 12:52:25.681643 13679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.667815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46261 (local address 127.0.0.1:40822)
0811 12:52:25.668051 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.668068 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:52:25.668183 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:52:25.668514 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:52:25.668519 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:52:25.668892 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:52:25.669080 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:25.669092 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.670244 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.670249 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:25.671107 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:25.671119 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.671912 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.671919 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:25.672154 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:25.672840 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:52:25.672863 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:52:25.675748 (+ 2885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:52:25.679321 (+ 3573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:25.679326 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:25.679331 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:25.679660 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:25.679997 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:25.680001 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:25.680004 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:25.680147 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:25.680637 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:52:25.680641 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:52:25.680771 (+ 130us) client_negotiation.cc:769] Sending connection context
0811 12:52:25.680902 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 12:52:25.681186 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141}
I20260811 12:52:25.682022 13677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.667727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40822 (local address 127.7.111.124:46261)
0811 12:52:25.667899 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:25.667904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:25.668137 (+ 233us) server_negotiation.cc:407] Connection header received
0811 12:52:25.668317 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:25.668323 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:25.668387 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:25.668461 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:25.669220 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.670128 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.671241 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.671755 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.671881 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:25.675944 (+ 4063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:25.675951 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:25.675955 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:25.675992 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:25.679200 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.679758 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.679763 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.679767 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.679874 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.680269 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.680274 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.680278 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.680531 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:25.680653 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:25.681004 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 12:52:25.681356 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":122,"server-negotiator.queue_time_us":60}
I20260811 12:52:25.686437 13541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.669746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39260 (local address 127.7.111.125:39411)
0811 12:52:25.669897 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:25.669902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:25.669934 (+ 32us) server_negotiation.cc:407] Connection header received
0811 12:52:25.670086 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:25.670092 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:25.670156 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:25.670232 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:25.671466 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.672443 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.674184 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.674817 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.674944 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:25.680197 (+ 5253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:25.680204 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:25.680208 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:25.680249 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:25.683555 (+ 3306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.684322 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.684328 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.684331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.684500 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.684895 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.684901 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.684905 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.685258 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:25.685545 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:25.685865 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 12:52:25.686101 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:52:25.687150 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.671880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46261 (local address 127.0.0.1:40824)
0811 12:52:25.672030 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.672049 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:52:25.672144 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:52:25.673505 (+ 1361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:52:25.673509 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:52:25.673916 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:52:25.674123 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:25.674136 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.675378 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.675384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:25.676506 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:25.676519 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.677317 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.677323 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:25.677548 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:25.678110 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:52:25.678132 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:52:25.680594 (+ 2462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:52:25.684597 (+ 4003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:25.684602 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:25.684605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:25.685029 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:25.685539 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:25.685544 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:25.685547 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:25.685692 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:25.686350 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:52:25.686355 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:52:25.686536 (+ 181us) client_negotiation.cc:769] Sending connection context
0811 12:52:25.686685 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 12:52:25.686881 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 12:52:25.687816 13674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd7467a76edf4c288a080e558cdb8cac: Bootstrap starting.
I20260811 12:52:25.687935 13683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.672202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40824 (local address 127.7.111.124:46261)
0811 12:52:25.673213 (+ 1011us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:25.673217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:25.673235 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:52:25.673278 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:25.673282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:25.673329 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:25.673450 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:25.674284 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.675243 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.676643 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.677171 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.677288 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:25.680872 (+ 3584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:25.680879 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:25.680884 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:25.680922 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:25.684437 (+ 3515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.685193 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.685197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.685204 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.685371 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.685840 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.685844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.685847 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.686206 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:25.687372 (+ 1166us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:25.687481 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 12:52:25.687632 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":903,"thread_start_us":684,"threads_started":1}
I20260811 12:52:25.690949 13674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd7467a76edf4c288a080e558cdb8cac: Neither blocks nor log segments found. Creating new log.
I20260811 12:52:25.692211 13674 log.cc:824] T 00000000000000000000000000000000 P dd7467a76edf4c288a080e558cdb8cac: Log is configured to *not* fsync() on all Append() calls
I20260811 12:52:25.692225 13402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c: Bootstrap starting.
I20260811 12:52:25.695708 13674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd7467a76edf4c288a080e558cdb8cac: No bootstrap required, opened a new log
I20260811 12:52:25.696053 13402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c: Neither blocks nor log segments found. Creating new log.
I20260811 12:52:25.697299 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:52:25.697356 13402 log.cc:824] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c: Log is configured to *not* fsync() on all Append() calls
I20260811 12:52:25.699922 13402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c: No bootstrap required, opened a new log
I20260811 12:52:25.700932 13355 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42952
I20260811 12:52:25.704017 13674 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd7467a76edf4c288a080e558cdb8cac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 45777 } } peers { permanent_uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } } peers { permanent_uuid: "dd7467a76edf4c288a080e558cdb8cac" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } }
I20260811 12:52:25.704330 13674 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd7467a76edf4c288a080e558cdb8cac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7467a76edf4c288a080e558cdb8cac, State: Initialized, Role: FOLLOWER
I20260811 12:52:25.704733 13674 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd7467a76edf4c288a080e558cdb8cac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 45777 } } peers { permanent_uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } } peers { permanent_uuid: "dd7467a76edf4c288a080e558cdb8cac" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } }
I20260811 12:52:25.705928 13686 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd7467a76edf4c288a080e558cdb8cac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 45777 } } peers { permanent_uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } } peers { permanent_uuid: "dd7467a76edf4c288a080e558cdb8cac" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } } }
I20260811 12:52:25.706537 13674 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd7467a76edf4c288a080e558cdb8cac [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:52:25.706658 13686 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd7467a76edf4c288a080e558cdb8cac [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:52:25.709324 13402 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 45777 } } peers { permanent_uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } } peers { permanent_uuid: "dd7467a76edf4c288a080e558cdb8cac" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } }
I20260811 12:52:25.709786 13402 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bc1b5df7ed14c0ab09ec51328743c5c, State: Initialized, Role: FOLLOWER
I20260811 12:52:25.710286 13402 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 45777 } } peers { permanent_uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } } peers { permanent_uuid: "dd7467a76edf4c288a080e558cdb8cac" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } }
I20260811 12:52:25.719913 13402 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:52:25.719990 13355 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
W20260811 12:52:25.720009 13712 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dd7467a76edf4c288a080e558cdb8cac: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:52:25.720118 13712 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:52:25.720193 13355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:52:25.723626 13355 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 45777 } } peers { permanent_uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } } peers { permanent_uuid: "dd7467a76edf4c288a080e558cdb8cac" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } }
I20260811 12:52:25.714129 13547 master_runner.cc:186] Error getting master registration for 127.7.111.126:45777: OK, instance_id { permanent_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" instance_seqno: 1786452744534782 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:52:25.725000 13547 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 12:52:25.725234 13355 leader_election.cc:288] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c [CANDIDATE]: Term 1 election: Requested vote from peers cab83f2fcfc549a8bf4b4defb4e7db3d (127.7.111.125:39411), dd7467a76edf4c288a080e558cdb8cac (127.7.111.124:46261)
I20260811 12:52:25.726267 13630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dd7467a76edf4c288a080e558cdb8cac"
I20260811 12:52:25.726794 13630 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd7467a76edf4c288a080e558cdb8cac [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:52:25.732098 13630 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dd7467a76edf4c288a080e558cdb8cac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bc1b5df7ed14c0ab09ec51328743c5c in term 1.
I20260811 12:52:25.733112 13305 leader_election.cc:302] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bc1b5df7ed14c0ab09ec51328743c5c, dd7467a76edf4c288a080e558cdb8cac; no voters:
I20260811 12:52:25.740762 13706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 45777 } } peers { permanent_uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } } peers { permanent_uuid: "dd7467a76edf4c288a080e558cdb8cac" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } } }
I20260811 12:52:25.742926 13706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:52:25.743916 13706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:52:25.744751 13706 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c [term 1 LEADER]: Becoming Leader. State: Replica: 4bc1b5df7ed14c0ab09ec51328743c5c, State: Running, Role: LEADER
I20260811 12:52:25.747679 13706 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 45777 } } peers { permanent_uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } } peers { permanent_uuid: "dd7467a76edf4c288a080e558cdb8cac" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } }
I20260811 12:52:25.764856 13722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bc1b5df7ed14c0ab09ec51328743c5c. Latest consensus state: current_term: 1 leader_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 45777 } } peers { permanent_uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } } peers { permanent_uuid: "dd7467a76edf4c288a080e558cdb8cac" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } } }
I20260811 12:52:25.765360 13722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c [sys.catalog]: This master's current role is: LEADER
W20260811 12:52:25.775332 13309 tablet.cc:2369] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:52:25.784416 13739 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:52:25.789922 13535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } attempt: 5
I20260811 12:52:25.795090 13541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.762861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39274 (local address 127.7.111.125:39411)
0811 12:52:25.763033 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:25.763039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:25.774016 (+ 10977us) server_negotiation.cc:407] Connection header received
0811 12:52:25.774303 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:25.774309 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:25.774393 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:25.774465 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:25.776671 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.777769 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.779696 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.780354 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.780459 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:25.785555 (+ 5096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:25.785564 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:25.785568 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:25.785620 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:25.789161 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.792519 (+ 3358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.792525 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.792528 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.792727 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.793191 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.793197 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.793201 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.793658 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:25.793886 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:25.794491 (+ 605us) server_negotiation.cc:299] Negotiation successful
0811 12:52:25.794721 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:52:25.798681 13683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.758560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40834 (local address 127.7.111.124:46261)
0811 12:52:25.758792 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:25.758799 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:25.767923 (+ 9124us) server_negotiation.cc:407] Connection header received
0811 12:52:25.769982 (+ 2059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:25.769989 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:25.770103 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:25.770180 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:25.775114 (+ 4934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.776195 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.781970 (+ 5775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.782621 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.782730 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:25.789189 (+ 6459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:25.789199 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:25.789228 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:25.789362 (+ 134us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:25.793533 (+ 4171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.794965 (+ 1432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.794971 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.794975 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.795155 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.795891 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.795897 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.795901 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.796307 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:25.796564 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:25.797597 (+ 1033us) server_negotiation.cc:299] Negotiation successful
0811 12:52:25.798011 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 12:52:25 dist-test-slave-f0tn krb5kdc[13251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452745, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.125@KRBTEST.COM for kudu/127.7.111.124@KRBTEST.COM
I20260811 12:52:25.810609 13678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.759648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42960 (local address 127.7.111.126:45777)
0811 12:52:25.759829 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:25.759835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:25.769504 (+ 9669us) server_negotiation.cc:407] Connection header received
0811 12:52:25.772500 (+ 2996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:25.772506 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:25.772586 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:25.772656 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:25.778388 (+ 5732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.779700 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.783706 (+ 4006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.784390 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.784495 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:25.787918 (+ 3423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:25.787925 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:25.787930 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:25.787994 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:25.798459 (+ 10465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.804071 (+ 5612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.804079 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.804083 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.804302 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.804812 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.804817 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.804821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.805214 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:25.805580 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:25.809853 (+ 4273us) server_negotiation.cc:299] Negotiation successful
0811 12:52:25.810116 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:52:25.814134 13677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.791324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40838 (local address 127.7.111.124:46261)
0811 12:52:25.791531 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:25.791536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:25.791555 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:52:25.791615 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:25.791621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:25.791689 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:25.791789 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:25.792893 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.793984 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.795866 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.796530 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.796650 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:25.805713 (+ 9063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:25.805720 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:25.805725 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:25.805767 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:25.809502 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.810842 (+ 1340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.810850 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.810854 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.811166 (+ 312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:25.811858 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:25.811864 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:25.811868 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:25.812524 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:25.812773 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:25.813555 (+ 782us) server_negotiation.cc:299] Negotiation successful
0811 12:52:25.813791 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:52:25.814404 13538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:25.790534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:46261 (local address 127.0.0.1:40838)
0811 12:52:25.790989 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:25.791010 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:52:25.791111 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:52:25.791986 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:52:25.791990 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:52:25.792442 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:52:25.792659 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:25.792671 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.794184 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.794190 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:25.795550 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:25.795565 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:25.800769 (+ 5204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:25.800776 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:25.801030 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:25.801650 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:52:25.801675 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:52:25.804827 (+ 3152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:52:25.809696 (+ 4869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:25.809703 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:25.809708 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:25.810647 (+ 939us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:25.811399 (+ 752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:25.811405 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:25.811410 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:25.811668 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:25.812917 (+ 1249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:52:25.812923 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:52:25.813180 (+ 257us) client_negotiation.cc:769] Sending connection context
0811 12:52:25.813440 (+ 260us) client_negotiation.cc:240] Negotiation successful
0811 12:52:25.813836 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334}
I20260811 12:52:25.817082 13739 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:52:25.821527 13535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cab83f2fcfc549a8bf4b4defb4e7db3d: Bootstrap starting.
I20260811 12:52:25.825829 13535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cab83f2fcfc549a8bf4b4defb4e7db3d: Neither blocks nor log segments found. Creating new log.
I20260811 12:52:25.827258 13535 log.cc:824] T 00000000000000000000000000000000 P cab83f2fcfc549a8bf4b4defb4e7db3d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:52:25.832911 13535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cab83f2fcfc549a8bf4b4defb4e7db3d: No bootstrap required, opened a new log
I20260811 12:52:25.840543 13630 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dd7467a76edf4c288a080e558cdb8cac [term 1 FOLLOWER]: Refusing update from remote peer 4bc1b5df7ed14c0ab09ec51328743c5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:52:25.841780 13722 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7467a76edf4c288a080e558cdb8cac" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 12:52:25.853775 13592 tablet.cc:2369] T 00000000000000000000000000000000 P dd7467a76edf4c288a080e558cdb8cac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:52:25.868578 13535 raft_consensus.cc:348] T 00000000000000000000000000000000 P cab83f2fcfc549a8bf4b4defb4e7db3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 45777 } } peers { permanent_uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } } peers { permanent_uuid: "dd7467a76edf4c288a080e558cdb8cac" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } }
I20260811 12:52:25.869064 13535 raft_consensus.cc:729] T 00000000000000000000000000000000 P cab83f2fcfc549a8bf4b4defb4e7db3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cab83f2fcfc549a8bf4b4defb4e7db3d, State: Initialized, Role: FOLLOWER
I20260811 12:52:25.876632 13757 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 12:52:25.885690 13535 consensus_queue.cc:260] T 00000000000000000000000000000000 P cab83f2fcfc549a8bf4b4defb4e7db3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 45777 } } peers { permanent_uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } } peers { permanent_uuid: "dd7467a76edf4c288a080e558cdb8cac" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } }
I20260811 12:52:25.890434 13686 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd7467a76edf4c288a080e558cdb8cac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bc1b5df7ed14c0ab09ec51328743c5c. Latest consensus state: current_term: 1 leader_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 45777 } } peers { permanent_uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } } peers { permanent_uuid: "dd7467a76edf4c288a080e558cdb8cac" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } } }
I20260811 12:52:25.890782 13686 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd7467a76edf4c288a080e558cdb8cac [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:52:25.881189 13282 master_runner.cc:186] Error getting master registration for 127.7.111.125:39411: OK, instance_id { permanent_uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d" instance_seqno: 1786452745064447 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:52:25.892388 13282 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 12:52:25.894910 13747 mvcc.cc:205] Tried to move back new op lower bound from 7317310446951841792 to 7317310446589153280. Current Snapshot: MvccSnapshot[applied={T|T < 7317310446951841792}]
I20260811 12:52:25.908332 13761 sys_catalog.cc:455] T 00000000000000000000000000000000 P cab83f2fcfc549a8bf4b4defb4e7db3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 45777 } } peers { permanent_uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } } peers { permanent_uuid: "dd7467a76edf4c288a080e558cdb8cac" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } } }
I20260811 12:52:25.909039 13761 sys_catalog.cc:458] T 00000000000000000000000000000000 P cab83f2fcfc549a8bf4b4defb4e7db3d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:52:25.911464 13485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cab83f2fcfc549a8bf4b4defb4e7db3d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:52:25.915167 13485 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cab83f2fcfc549a8bf4b4defb4e7db3d [term 1 FOLLOWER]: Refusing update from remote peer 4bc1b5df7ed14c0ab09ec51328743c5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:52:25.915781 13486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d"
I20260811 12:52:25.916352 13722 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:52:25.917259 13722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 45777 } } peers { permanent_uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } } peers { permanent_uuid: "dd7467a76edf4c288a080e558cdb8cac" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } } }
I20260811 12:52:25.917528 13722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:25.919235 13486 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cab83f2fcfc549a8bf4b4defb4e7db3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bc1b5df7ed14c0ab09ec51328743c5c in term 1.
I20260811 12:52:25.920217 13739 catalog_manager.cc:1347] Generated new cluster ID: c0026c9990f84033993678d89266bf10
I20260811 12:52:25.920326 13739 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:52:25.923702 13535 sys_catalog.cc:564] T 00000000000000000000000000000000 P cab83f2fcfc549a8bf4b4defb4e7db3d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:52:25.934707 13686 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd7467a76edf4c288a080e558cdb8cac [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 45777 } } peers { permanent_uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } } peers { permanent_uuid: "dd7467a76edf4c288a080e558cdb8cac" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } } }
I20260811 12:52:25.935046 13686 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd7467a76edf4c288a080e558cdb8cac [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:52:25.971055 13774 sys_catalog.cc:455] T 00000000000000000000000000000000 P cab83f2fcfc549a8bf4b4defb4e7db3d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 45777 } } peers { permanent_uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } } peers { permanent_uuid: "dd7467a76edf4c288a080e558cdb8cac" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } } }
I20260811 12:52:25.971637 13774 sys_catalog.cc:458] T 00000000000000000000000000000000 P cab83f2fcfc549a8bf4b4defb4e7db3d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:52:25.972204 13761 sys_catalog.cc:455] T 00000000000000000000000000000000 P cab83f2fcfc549a8bf4b4defb4e7db3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bc1b5df7ed14c0ab09ec51328743c5c. Latest consensus state: current_term: 1 leader_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 45777 } } peers { permanent_uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } } peers { permanent_uuid: "dd7467a76edf4c288a080e558cdb8cac" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } } }
I20260811 12:52:25.972559 13761 sys_catalog.cc:458] T 00000000000000000000000000000000 P cab83f2fcfc549a8bf4b4defb4e7db3d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:52:25.986372 13706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc1b5df7ed14c0ab09ec51328743c5c" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 45777 } } peers { permanent_uuid: "cab83f2fcfc549a8bf4b4defb4e7db3d" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 39411 } } peers { permanent_uuid: "dd7467a76edf4c288a080e558cdb8cac" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 46261 } } }
I20260811 12:52:25.986752 13706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:25.994741 13739 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:52:25.998355 13739 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:52:26.029964 13798 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 12:52:26.030339 13739 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4bc1b5df7ed14c0ab09ec51328743c5c: Generated new TSK 0
I20260811 12:52:26.032202 13739 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 12:52:26.136296 13678 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:26.127397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42976 (local address 127.7.111.126:45777)
0811 12:52:26.127803 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:26.127810 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:26.133818 (+ 6008us) server_negotiation.cc:407] Connection header received
0811 12:52:26.134383 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:26.134391 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:26.134496 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:26.134588 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:26.136014 (+ 1426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42976: BlockingRecv error: recv got EOF from 127.0.0.1:42976 (error 108)
Metrics: {"server-negotiator.queue_time_us":274}
W20260811 12:52:26.139292 13541 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:26.126475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39290 (local address 127.7.111.125:39411)
0811 12:52:26.126708 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:26.126715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:26.137371 (+ 10656us) server_negotiation.cc:407] Connection header received
0811 12:52:26.137692 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:26.137698 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:26.137811 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:26.137884 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:26.139027 (+ 1143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39290: BlockingRecv error: recv got EOF from 127.0.0.1:39290 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260811 12:52:26.142381 13677 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:26.127603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40840 (local address 127.7.111.124:46261)
0811 12:52:26.127961 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:26.127967 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:26.140415 (+ 12448us) server_negotiation.cc:407] Connection header received
0811 12:52:26.140696 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:26.140704 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:26.140797 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:26.140875 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:26.142101 (+ 1226us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40840: BlockingRecv error: recv got EOF from 127.0.0.1:40840 (error 108)
Metrics: {"server-negotiator.queue_time_us":230}
W20260811 12:52:26.150688 13541 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:26.148367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39298 (local address 127.7.111.125:39411)
0811 12:52:26.148577 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:26.148584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:26.149048 (+ 464us) server_negotiation.cc:407] Connection header received
0811 12:52:26.149300 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:26.149304 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:26.149469 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:26.149569 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:26.150389 (+ 820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39298: BlockingRecv error: recv got EOF from 127.0.0.1:39298 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 12:52:26.156661 13541 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:26.154735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39310 (local address 127.7.111.125:39411)
0811 12:52:26.154913 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:26.154919 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:26.155375 (+ 456us) server_negotiation.cc:407] Connection header received
0811 12:52:26.155580 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:26.155585 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:26.155654 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:26.155734 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:26.156443 (+ 709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39310: BlockingRecv error: recv got EOF from 127.0.0.1:39310 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 12:52:26.163542 13541 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:26.161502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39326 (local address 127.7.111.125:39411)
0811 12:52:26.161695 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:26.161701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:26.162216 (+ 515us) server_negotiation.cc:407] Connection header received
0811 12:52:26.162403 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:26.162409 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:26.162484 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:26.162576 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:26.163297 (+ 721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39326: BlockingRecv error: recv got EOF from 127.0.0.1:39326 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 12:52:26.176790 13541 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:26.174667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39332 (local address 127.7.111.125:39411)
0811 12:52:26.174921 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:26.174929 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:26.175435 (+ 506us) server_negotiation.cc:407] Connection header received
0811 12:52:26.175630 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:26.175636 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:26.175713 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:26.175799 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:26.176554 (+ 755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39332: BlockingRecv error: recv got EOF from 127.0.0.1:39332 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
W20260811 12:52:26.195775 13541 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:26.188133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39348 (local address 127.7.111.125:39411)
0811 12:52:26.188323 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:26.188329 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:26.193815 (+ 5486us) server_negotiation.cc:407] Connection header received
0811 12:52:26.194149 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:26.194157 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:26.194270 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:26.194357 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:26.195537 (+ 1180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39348: BlockingRecv error: recv got EOF from 127.0.0.1:39348 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 12:52:26.218542 13541 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:26.216081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39356 (local address 127.7.111.125:39411)
0811 12:52:26.216295 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:26.216302 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:26.216885 (+ 583us) server_negotiation.cc:407] Connection header received
0811 12:52:26.217108 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:26.217115 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:26.217221 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:26.217360 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:26.218281 (+ 921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39356: BlockingRecv error: recv got EOF from 127.0.0.1:39356 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260811 12:52:26.262643 13541 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:26.259418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39368 (local address 127.7.111.125:39411)
0811 12:52:26.259610 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:26.259616 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:26.260213 (+ 597us) server_negotiation.cc:407] Connection header received
0811 12:52:26.260432 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:26.260439 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:26.260550 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:26.260677 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:26.261677 (+ 1000us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39368: BlockingRecv error: recv got EOF from 127.0.0.1:39368 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 12:52:26.331328 13541 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:26.329320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39376 (local address 127.7.111.125:39411)
0811 12:52:26.329574 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:26.329582 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:26.329833 (+ 251us) server_negotiation.cc:407] Connection header received
0811 12:52:26.330040 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:26.330047 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:26.330152 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:26.330284 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:26.331060 (+ 776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39376: BlockingRecv error: recv got EOF from 127.0.0.1:39376 (error 108)
Metrics: {"server-negotiator.queue_time_us":119}
W20260811 12:52:26.465157 13541 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:26.463122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39380 (local address 127.7.111.125:39411)
0811 12:52:26.463304 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:26.463310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:26.463821 (+ 511us) server_negotiation.cc:407] Connection header received
0811 12:52:26.464013 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:26.464017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:26.464090 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:26.464171 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:26.464950 (+ 779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39380: BlockingRecv error: recv got EOF from 127.0.0.1:39380 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:52:26.722803 13712 catalog_manager.cc:1259] Loaded cluster ID: c0026c9990f84033993678d89266bf10
I20260811 12:52:26.722936 13712 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dd7467a76edf4c288a080e558cdb8cac: loading cluster ID for follower catalog manager: success
I20260811 12:52:26.726841 13712 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dd7467a76edf4c288a080e558cdb8cac: acquiring CA information for follower catalog manager: success
I20260811 12:52:26.729282 13712 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dd7467a76edf4c288a080e558cdb8cac: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 12:52:26.729943 13541 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:26.727614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39390 (local address 127.7.111.125:39411)
0811 12:52:26.727820 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:26.727827 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:26.728430 (+ 603us) server_negotiation.cc:407] Connection header received
0811 12:52:26.728628 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:26.728632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:26.728721 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:26.728808 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:26.729738 (+ 930us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39390: BlockingRecv error: recv got EOF from 127.0.0.1:39390 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 12:52:27.034579 13798 catalog_manager.cc:1259] Loaded cluster ID: c0026c9990f84033993678d89266bf10
I20260811 12:52:27.034720 13798 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cab83f2fcfc549a8bf4b4defb4e7db3d: loading cluster ID for follower catalog manager: success
I20260811 12:52:27.037902 13798 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cab83f2fcfc549a8bf4b4defb4e7db3d: acquiring CA information for follower catalog manager: success
I20260811 12:52:27.039973 13798 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cab83f2fcfc549a8bf4b4defb4e7db3d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:52:27.257417 13910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:27.250796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39400 (local address 127.7.111.125:39411)
0811 12:52:27.251661 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:27.251666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:27.251688 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:52:27.251755 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:27.251758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:27.251908 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:27.252028 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:52:27.252511 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:27.253494 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:27.254853 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:27.255341 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:27.255456 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:27.256038 (+ 582us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:52:27.256671 (+ 633us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:52:27.256819 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:27.256940 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 12:52:27.257023 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":728,"thread_start_us":520,"threads_started":1}
I20260811 12:52:27.269562 13920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:27.263807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40850 (local address 127.7.111.124:46261)
0811 12:52:27.264453 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:27.264459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:27.264477 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:52:27.264576 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:27.264579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:27.264738 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:27.264891 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:52:27.265301 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:27.266030 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:27.267334 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:27.267754 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:27.267867 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:27.268344 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:52:27.268879 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:52:27.269028 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:27.269123 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 12:52:27.269222 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":396,"threads_started":1}
I20260811 12:52:27.287477 13910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:27.276144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39404 (local address 127.7.111.125:39411)
0811 12:52:27.276732 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:27.276738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:27.278348 (+ 1610us) server_negotiation.cc:407] Connection header received
0811 12:52:27.278497 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:27.278503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:27.278602 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:27.278735 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:52:27.279230 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:27.280360 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:27.283202 (+ 2842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:27.283896 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:27.283996 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:27.286492 (+ 2496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:52:27.286628 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:52:27.286704 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:27.287019 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 12:52:27.287111 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435}
I20260811 12:52:27.289162 13934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:27.276908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42988 (local address 127.7.111.126:45777)
0811 12:52:27.281811 (+ 4903us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:27.281817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:27.281838 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:52:27.281957 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:27.281963 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:27.282206 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:27.282408 (+ 202us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:52:27.283128 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:27.284174 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:27.286217 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:27.286816 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:27.287445 (+ 629us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:27.287650 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:52:27.288122 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:52:27.288749 (+ 627us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:27.288800 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 12:52:27.288883 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4752,"thread_start_us":4646,"threads_started":1}
I20260811 12:52:27.290663 13920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:27.277054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40854 (local address 127.7.111.124:46261)
0811 12:52:27.277233 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:27.277238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:27.283189 (+ 5951us) server_negotiation.cc:407] Connection header received
0811 12:52:27.283631 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:27.283638 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:27.283789 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:27.283862 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:52:27.284728 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:27.285902 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:27.288463 (+ 2561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:27.289082 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:27.289189 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:27.289975 (+ 786us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:52:27.290110 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:52:27.290208 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:27.290326 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 12:52:27.290410 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 12:52:27.295925 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 13282
I20260811 12:52:27.297063 13397 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:52:27.409243 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 13282
I20260811 12:52:27.441252 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 13414
I20260811 12:52:27.442421 13525 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:52:27.549691 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 13414
I20260811 12:52:27.578491 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 13547
I20260811 12:52:27.579635 13665 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:52:27.687889 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 13547
2026-08-11T12:52:27Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5225 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:52:27 dist-test-slave-f0tn krb5kdc[13944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:52:27 dist-test-slave-f0tn krb5kdc[13944](info): set up 2 sockets
Aug 11 12:52:27 dist-test-slave-f0tn krb5kdc[13944](info): commencing operation
krb5kdc: starting...
W20260811 12:52:28.985913 7613 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.005s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/test-user.keytab.
Aug 11 12:52:29 dist-test-slave-f0tn krb5kdc[13944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:52:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:52:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.111.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
Entry for principal kudu/127.7.111.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab.
I20260811 12:52:29.342880 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:40439
--webserver_interface=127.7.111.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:35235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:40439,127.7.111.125:37569,127.7.111.124:37447
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:52:29.687721 13966 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:52:29.688036 13966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:52:29.688134 13966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:52:29.688231 13966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:52:29.695554 13966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:52:29.695640 13966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:52:29.695712 13966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:52:29.695780 13966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:52:29.695864 13966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:52:29.705612 13966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:35235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:40439,127.7.111.125:37569,127.7.111.124:37447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.126.keytab
--principal=kudu/127.7.111.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.126:40439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.111.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:52:29.707022 13966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:52:29.708357 13966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:52:29.719648 13980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:29.719708 13974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:29.719736 13978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:52:29.720672 13966 server_base.cc:1034] running on GCE node
I20260811 12:52:29.721480 13966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:52:29.722819 13966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:52:29.724028 13966 hybrid_clock.cc:648] HybridClock initialized: now 1786452749723960 us; error 48 us; skew 500 ppm
I20260811 12:52:29.724344 13966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:52:29 dist-test-slave-f0tn krb5kdc[13944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452749, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:52:29.727463 13966 init.cc:377] Logged in from keytab as kudu/127.7.111.126@KRBTEST.COM (short username kudu)
I20260811 12:52:29.729100 13966 webserver.cc:460] Webserver started at http://127.7.111.126:46365/ using document root <none> and password file <none>
I20260811 12:52:29.730003 13966 fs_manager.cc:359] Metadata directory not provided
I20260811 12:52:29.730134 13966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:52:29.730511 13966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:52:29.733743 13966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-0/data/instance:
uuid: "09bd13ff4b654bb8835e2cf73666d1f3"
format_stamp: "Formatted at 2026-08-11 12:52:29 on dist-test-slave-f0tn"
server_key: "9f7c48064843c0a570868f5099dd17c6"
server_key_iv: "2b9bc4b3e6de6fd0929876834b70631a"
server_key_version: "encryptionkey@0"
I20260811 12:52:29.734506 13966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-0/wal/instance:
uuid: "09bd13ff4b654bb8835e2cf73666d1f3"
format_stamp: "Formatted at 2026-08-11 12:52:29 on dist-test-slave-f0tn"
server_key: "9f7c48064843c0a570868f5099dd17c6"
server_key_iv: "2b9bc4b3e6de6fd0929876834b70631a"
server_key_version: "encryptionkey@0"
I20260811 12:52:29.739709 13966 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:52:29.743091 13990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:52:29.744371 13966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:52:29.744525 13966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-0/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-0/wal
uuid: "09bd13ff4b654bb8835e2cf73666d1f3"
format_stamp: "Formatted at 2026-08-11 12:52:29 on dist-test-slave-f0tn"
server_key: "9f7c48064843c0a570868f5099dd17c6"
server_key_iv: "2b9bc4b3e6de6fd0929876834b70631a"
server_key_version: "encryptionkey@0"
I20260811 12:52:29.744725 13966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:52:29.755352 13966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:52:29.758970 13966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:52:29.759210 13966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:52:29.776508 13966 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.126:40439
I20260811 12:52:29.776525 14081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.126:40439 every 8 connection(s)
I20260811 12:52:29.777915 13966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-0/data/info.pb
I20260811 12:52:29.781528 14085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:52:29.786989 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 13966
I20260811 12:52:29.787169 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-0/wal/instance
I20260811 12:52:29.787585 7613 external_mini_cluster.cc:1428] Setting key b556622c6269ea8f5aaca57ab3f73dec
I20260811 12:52:29.787842 14085 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40439 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:52:29 dist-test-slave-f0tn krb5kdc[13944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452749, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:52:29.809774 14088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:29.791949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34656 (local address 127.7.111.126:40439)
0811 12:52:29.792603 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:29.792611 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:29.792664 (+ 53us) server_negotiation.cc:407] Connection header received
0811 12:52:29.793900 (+ 1236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:29.793907 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:29.794212 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:29.794416 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:29.795547 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:29.797175 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:29.798568 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:29.799333 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:29.799505 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:29.802780 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:29.802789 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:29.802803 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:29.802842 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:29.805671 (+ 2829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:29.806463 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:29.806470 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:29.806472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:29.806622 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:29.807039 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:29.807042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:29.807044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:29.807523 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:29.807679 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:29.808396 (+ 717us) server_negotiation.cc:299] Negotiation successful
0811 12:52:29.808637 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":460,"thread_start_us":314,"threads_started":1}
I20260811 12:52:29.809818 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:29.790731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40439 (local address 127.0.0.1:34656)
0811 12:52:29.791497 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:29.791550 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 12:52:29.793548 (+ 1998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:52:29.794453 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:52:29.794458 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:52:29.794992 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:52:29.795345 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:29.795362 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:29.797346 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:29.797349 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:29.798433 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:29.798446 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:29.799534 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:29.799539 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:29.799680 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:29.800270 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:52:29.800293 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:52:29.802555 (+ 2262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:52:29.805800 (+ 3245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:29.805813 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:29.805826 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:29.806339 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:29.806746 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:29.806750 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:29.806753 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:29.806946 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:29.807662 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:52:29.807670 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:52:29.807926 (+ 256us) client_negotiation.cc:769] Sending connection context
0811 12:52:29.808249 (+ 323us) client_negotiation.cc:240] Negotiation successful
0811 12:52:29.808808 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":91,"thread_start_us":306,"threads_started":1}
I20260811 12:52:29.813086 14085 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:52:29.814494 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:29.814064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:37569 (local address 127.0.0.1:59778)
0811 12:52:29.814244 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:29.814374 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:37569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260811 12:52:29.814868 13993 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.111.125:37569: connect: Connection refused (error 111)
I20260811 12:52:29.816581 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:29.816238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:37569 (local address 127.0.0.1:59780)
0811 12:52:29.816380 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:29.816450 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:37569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 12:52:29.816802 14085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.125:37569: Network error: Client connection negotiation failed: client connection to 127.7.111.125:37569: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.7.111.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.125.keytab.
Entry for principal kudu/127.7.111.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.125.keytab.
I20260811 12:52:29.864665 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.125:37569
--webserver_interface=127.7.111.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.125.keytab
--principal=kudu/127.7.111.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:35235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:40439,127.7.111.125:37569,127.7.111.124:37447
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:52:29.866134 14085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } attempt: 1
I20260811 12:52:29.867257 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:29.866794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:37569 (local address 127.0.0.1:59782)
0811 12:52:29.866992 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:29.867099 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:37569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 12:52:29.868288 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:29.867942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:37569 (local address 127.0.0.1:59786)
0811 12:52:29.868089 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:29.868152 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:37569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 12:52:29.868458 14085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.125:37569: Network error: Client connection negotiation failed: client connection to 127.7.111.125:37569: connect: Connection refused (error 111)
I20260811 12:52:29.936712 14085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } attempt: 2
I20260811 12:52:29.938024 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:29.937447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:37569 (local address 127.0.0.1:59802)
0811 12:52:29.937713 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:29.937839 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:37569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 12:52:29.939073 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:29.938665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:37569 (local address 127.0.0.1:59818)
0811 12:52:29.938824 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:29.938924 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:37569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 12:52:29.939294 14085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.125:37569: Network error: Client connection negotiation failed: client connection to 127.7.111.125:37569: connect: Connection refused (error 111)
I20260811 12:52:30.030575 14085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } attempt: 3
I20260811 12:52:30.031879 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.031341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:37569 (local address 127.0.0.1:59824)
0811 12:52:30.031572 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.031694 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:37569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 12:52:30.032994 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.032543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:37569 (local address 127.0.0.1:59834)
0811 12:52:30.032711 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.032806 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:37569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 12:52:30.033216 14085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.125:37569: Network error: Client connection negotiation failed: client connection to 127.7.111.125:37569: connect: Connection refused (error 111)
I20260811 12:52:30.176499 14085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } attempt: 4
I20260811 12:52:30.177896 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.177192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:37569 (local address 127.0.0.1:59850)
0811 12:52:30.177585 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.177714 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:37569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225}
I20260811 12:52:30.179019 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.178596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:37569 (local address 127.0.0.1:59860)
0811 12:52:30.178751 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.178862 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.125:37569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 12:52:30.179226 14085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.125:37569: Network error: Client connection negotiation failed: client connection to 127.7.111.125:37569: connect: Connection refused (error 111)
W20260811 12:52:30.209710 14094 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:52:30.210021 14094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:52:30.210094 14094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:52:30.210175 14094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:52:30.217403 14094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:52:30.217505 14094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:52:30.217573 14094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:52:30.217644 14094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:52:30.217751 14094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:52:30.227692 14094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:35235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:40439,127.7.111.125:37569,127.7.111.124:37447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.125.keytab
--principal=kudu/127.7.111.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.125:37569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.111.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:52:30.229014 14094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:52:30.230420 14094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:52:30.241745 14106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:30.241779 14105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:30.241870 14109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:52:30.242709 14094 server_base.cc:1034] running on GCE node
I20260811 12:52:30.243436 14094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:52:30.244828 14094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:52:30.246058 14094 hybrid_clock.cc:648] HybridClock initialized: now 1786452750245975 us; error 50 us; skew 500 ppm
I20260811 12:52:30.246389 14094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:52:30 dist-test-slave-f0tn krb5kdc[13944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452750, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:52:30.253289 14094 init.cc:377] Logged in from keytab as kudu/127.7.111.125@KRBTEST.COM (short username kudu)
I20260811 12:52:30.254761 14094 webserver.cc:460] Webserver started at http://127.7.111.125:36267/ using document root <none> and password file <none>
I20260811 12:52:30.255467 14094 fs_manager.cc:359] Metadata directory not provided
I20260811 12:52:30.255573 14094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:52:30.255888 14094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:52:30.258932 14094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-1/data/instance:
uuid: "54b443bf04034383a9031f8b3c26c25c"
format_stamp: "Formatted at 2026-08-11 12:52:30 on dist-test-slave-f0tn"
server_key: "d29a08c73393d5203725eb2816404110"
server_key_iv: "108d3609b51a3c57b018e1821165cd13"
server_key_version: "encryptionkey@0"
I20260811 12:52:30.259711 14094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-1/wal/instance:
uuid: "54b443bf04034383a9031f8b3c26c25c"
format_stamp: "Formatted at 2026-08-11 12:52:30 on dist-test-slave-f0tn"
server_key: "d29a08c73393d5203725eb2816404110"
server_key_iv: "108d3609b51a3c57b018e1821165cd13"
server_key_version: "encryptionkey@0"
I20260811 12:52:30.264883 14094 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 12:52:30.268316 14125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:52:30.269655 14094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:52:30.269811 14094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-1/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-1/wal
uuid: "54b443bf04034383a9031f8b3c26c25c"
format_stamp: "Formatted at 2026-08-11 12:52:30 on dist-test-slave-f0tn"
server_key: "d29a08c73393d5203725eb2816404110"
server_key_iv: "108d3609b51a3c57b018e1821165cd13"
server_key_version: "encryptionkey@0"
I20260811 12:52:30.270010 14094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:52:30.298475 14094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:52:30.302843 14094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:52:30.303119 14094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:52:30.321148 14094 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.125:37569
I20260811 12:52:30.321169 14211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.125:37569 every 8 connection(s)
I20260811 12:52:30.322633 14094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-1/data/info.pb
I20260811 12:52:30.326547 14212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:52:30.328334 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 14094
I20260811 12:52:30.328510 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-1/wal/instance
I20260811 12:52:30.328955 7613 external_mini_cluster.cc:1428] Setting key f8b022ed19b9ff0a1d0fc1023c6a6b3a
I20260811 12:52:30.333561 14212 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40439 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:52:30 dist-test-slave-f0tn krb5kdc[13944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452750, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.125@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:52:30.360116 14214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.336390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34664 (local address 127.7.111.126:40439)
0811 12:52:30.337801 (+ 1411us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:30.337807 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:30.338369 (+ 562us) server_negotiation.cc:407] Connection header received
0811 12:52:30.340287 (+ 1918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:30.340293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:30.340388 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:30.340513 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:30.342589 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.343754 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.346120 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.346691 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.346819 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:30.353035 (+ 6216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:30.353043 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:30.353049 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:30.353107 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:30.356852 (+ 3745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:30.357772 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:30.357776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:30.357779 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:30.357906 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:30.358376 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:30.358378 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:30.358380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:30.358652 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:30.358772 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:30.359674 (+ 902us) server_negotiation.cc:299] Negotiation successful
0811 12:52:30.359846 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1268,"thread_start_us":1128,"threads_started":1}
I20260811 12:52:30.361609 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.337030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40439 (local address 127.0.0.1:34664)
0811 12:52:30.338180 (+ 1150us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.338245 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 12:52:30.340009 (+ 1764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:52:30.340699 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:52:30.340707 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:52:30.341489 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:52:30.342299 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:30.342327 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.343954 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.343959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:30.345921 (+ 1962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:30.345938 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.346980 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.346988 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:30.347293 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:30.348203 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:52:30.348247 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:52:30.352660 (+ 4413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:52:30.357011 (+ 4351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:30.357021 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:30.357040 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:30.357644 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:30.358024 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:30.358029 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:30.358032 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:30.358281 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:30.358775 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:52:30.358783 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:52:30.359261 (+ 478us) client_negotiation.cc:769] Sending connection context
0811 12:52:30.359624 (+ 363us) client_negotiation.cc:240] Negotiation successful
0811 12:52:30.360160 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":859,"thread_start_us":618,"threads_started":1}
I20260811 12:52:30.363456 14212 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.111.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.111.124@KRBTEST.COM" created.
Aug 11 12:52:30 dist-test-slave-f0tn krb5kdc[13944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452750, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.125@KRBTEST.COM for kudu/127.7.111.125@KRBTEST.COM
I20260811 12:52:30.377877 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.364254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:37569 (local address 127.0.0.1:59864)
0811 12:52:30.364449 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.364469 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:52:30.364575 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:52:30.365585 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:52:30.365590 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:52:30.366036 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:52:30.366244 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:30.366257 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.367613 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.367617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:30.368453 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:30.368464 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.369329 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.369335 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:30.369521 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:30.370032 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:52:30.370050 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:52:30.371967 (+ 1917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:52:30.375508 (+ 3541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:30.375513 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:30.375516 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:30.375939 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:30.376326 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:30.376328 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:30.376351 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:30.376473 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:30.377125 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:52:30.377128 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:52:30.377259 (+ 131us) client_negotiation.cc:769] Sending connection context
0811 12:52:30.377365 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 12:52:30.377604 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 12:52:30.378160 14220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.364475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59864 (local address 127.7.111.125:37569)
0811 12:52:30.365131 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:30.365137 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:30.365163 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:52:30.365208 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:30.365214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:30.365422 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:30.365547 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:30.366418 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.367478 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.368571 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.369173 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.369296 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:30.372174 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:30.372187 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:30.372204 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:30.372238 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:30.375365 (+ 3127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:30.376062 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:30.376070 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:30.376072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:30.376205 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:30.376580 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:30.376585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:30.376587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:30.377009 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:30.377221 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:30.377542 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 12:52:30.377793 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":356,"threads_started":1}
I20260811 12:52:30.380581 14212 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:52:30.381798 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.381300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:37447 (local address 127.0.0.1:52422)
0811 12:52:30.381501 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.381607 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 12:52:30.382102 14130 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
I20260811 12:52:30.383572 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.383271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:37447 (local address 127.0.0.1:52430)
0811 12:52:30.383397 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.383476 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 12:52:30.383795 14212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:37447: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.111.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.124.keytab.
Entry for principal kudu/127.7.111.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.124.keytab.
I20260811 12:52:30.406353 7613 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
/tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.124:37447
--webserver_interface=127.7.111.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.124.keytab
--principal=kudu/127.7.111.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.111.84:35235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.111.126:40439,127.7.111.125:37569,127.7.111.124:37447
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:52:30.433109 14212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } attempt: 1
I20260811 12:52:30.434360 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.433884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:37447 (local address 127.0.0.1:52444)
0811 12:52:30.434088 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.434200 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 12:52:30.435377 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.435049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:37447 (local address 127.0.0.1:52448)
0811 12:52:30.435177 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.435268 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 12:52:30.435576 14212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:37447: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
I20260811 12:52:30.478477 14085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } attempt: 5
Aug 11 12:52:30 dist-test-slave-f0tn krb5kdc[13944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452749, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for kudu/127.7.111.125@KRBTEST.COM
I20260811 12:52:30.491410 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.479220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:37569 (local address 127.0.0.1:59880)
0811 12:52:30.479449 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.479471 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:52:30.479615 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:52:30.479994 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:52:30.479998 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:52:30.480404 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:52:30.480591 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:30.480601 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.481670 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.481674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:30.482549 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:30.482561 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.483306 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.483312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:30.483525 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:30.484128 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:52:30.484146 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:52:30.486159 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:52:30.489007 (+ 2848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:30.489013 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:30.489018 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:30.489430 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:30.489775 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:30.489780 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:30.489785 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:30.489956 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:30.490469 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:52:30.490474 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:52:30.490626 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 12:52:30.490743 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 12:52:30.491057 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 12:52:30.491412 14220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.479354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59880 (local address 127.7.111.125:37569)
0811 12:52:30.479552 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:30.479558 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:30.479581 (+ 23us) server_negotiation.cc:407] Connection header received
0811 12:52:30.479749 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:30.479753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:30.479840 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:30.479926 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:30.480752 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.481523 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.482701 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.483163 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.483275 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:30.486338 (+ 3063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:30.486343 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:30.486346 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:30.486379 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:30.488882 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:30.489541 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:30.489545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:30.489547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:30.489667 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:30.490059 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:30.490064 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:30.490068 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:30.490356 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:30.490490 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:30.490860 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 12:52:30.491087 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:52:30.492689 14085 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:52:30.493886 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.493498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:37447 (local address 127.0.0.1:52450)
0811 12:52:30.493678 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.493753 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 12:52:30.494911 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.494491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:37447 (local address 127.0.0.1:52462)
0811 12:52:30.494655 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.494776 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 12:52:30.495172 14085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:37447: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
I20260811 12:52:30.503778 14212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } attempt: 2
I20260811 12:52:30.504784 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.504342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:37447 (local address 127.0.0.1:52466)
0811 12:52:30.504550 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.504637 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 12:52:30.505856 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.505537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:37447 (local address 127.0.0.1:52482)
0811 12:52:30.505669 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.505740 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 12:52:30.506063 14212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:37447: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
I20260811 12:52:30.546432 14085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } attempt: 1
I20260811 12:52:30.547734 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.547168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:37447 (local address 127.0.0.1:52490)
0811 12:52:30.547394 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.547510 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 12:52:30.548830 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.548454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:37447 (local address 127.0.0.1:52506)
0811 12:52:30.548599 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.548689 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 12:52:30.549044 14085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:37447: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
I20260811 12:52:30.597268 14212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } attempt: 3
I20260811 12:52:30.598609 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.598077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:37447 (local address 127.0.0.1:52520)
0811 12:52:30.598329 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.598428 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 12:52:30.599795 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.599345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:37447 (local address 127.0.0.1:52534)
0811 12:52:30.599591 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.599674 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
W20260811 12:52:30.600008 14212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:37447: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
I20260811 12:52:30.617282 14085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } attempt: 2
I20260811 12:52:30.618721 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.618071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:37447 (local address 127.0.0.1:52542)
0811 12:52:30.618352 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.618489 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 12:52:30.619797 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.619453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:37447 (local address 127.0.0.1:52554)
0811 12:52:30.619599 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.619682 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 12:52:30.620031 14085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:37447: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
I20260811 12:52:30.726246 14085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } attempt: 3
I20260811 12:52:30.727340 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.726862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:37447 (local address 127.0.0.1:52568)
0811 12:52:30.727058 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.727151 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 12:52:30.728542 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.728147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:37447 (local address 127.0.0.1:52572)
0811 12:52:30.728358 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.728434 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260811 12:52:30.728760 14085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:37447: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
I20260811 12:52:30.743211 14212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } attempt: 4
I20260811 12:52:30.744328 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.743873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:37447 (local address 127.0.0.1:52580)
0811 12:52:30.744079 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.744164 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 12:52:30.745399 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.745034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:37447 (local address 127.0.0.1:52592)
0811 12:52:30.745195 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.745263 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 12:52:30.745661 14212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.111.124:37447: Network error: Client connection negotiation failed: client connection to 127.7.111.124:37447: connect: Connection refused (error 111)
W20260811 12:52:30.766312 14225 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:52:30.766649 14225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:52:30.766758 14225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:52:30.766886 14225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:52:30.774293 14225 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:52:30.774389 14225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:52:30.774492 14225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:52:30.774569 14225 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:52:30.774662 14225 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:52:30.784595 14225 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.111.84:35235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.111.126:40439,127.7.111.125:37569,127.7.111.124:37447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/krb5kdc/kudu_127.7.111.124.keytab
--principal=kudu/127.7.111.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.111.124:37447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.111.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 11:11:46 UTC on ccycloud.quasar-zjjjhb.root.comops.site
build id 4580142
ASAN enabled
I20260811 12:52:30.785882 14225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:52:30.787199 14225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:52:30.798715 14235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:30.798740 14233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:52:30.799238 14232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:52:30.799568 14225 server_base.cc:1034] running on GCE node
I20260811 12:52:30.800354 14225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:52:30.801785 14225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:52:30.802990 14225 hybrid_clock.cc:648] HybridClock initialized: now 1786452750802916 us; error 55 us; skew 500 ppm
I20260811 12:52:30.803323 14225 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:52:30 dist-test-slave-f0tn krb5kdc[13944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452750, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:52:30.806286 14225 init.cc:377] Logged in from keytab as kudu/127.7.111.124@KRBTEST.COM (short username kudu)
I20260811 12:52:30.807685 14225 webserver.cc:460] Webserver started at http://127.7.111.124:37079/ using document root <none> and password file <none>
I20260811 12:52:30.808353 14225 fs_manager.cc:359] Metadata directory not provided
I20260811 12:52:30.808449 14225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:52:30.808751 14225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:52:30.811796 14225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-2/data/instance:
uuid: "770cde8817b746d88511047f98019f0a"
format_stamp: "Formatted at 2026-08-11 12:52:30 on dist-test-slave-f0tn"
server_key: "0d2c906f0309f2b86da12e69c8170687"
server_key_iv: "9afffe42270757f1cd6964390cc0d233"
server_key_version: "encryptionkey@0"
I20260811 12:52:30.812609 14225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-2/wal/instance:
uuid: "770cde8817b746d88511047f98019f0a"
format_stamp: "Formatted at 2026-08-11 12:52:30 on dist-test-slave-f0tn"
server_key: "0d2c906f0309f2b86da12e69c8170687"
server_key_iv: "9afffe42270757f1cd6964390cc0d233"
server_key_version: "encryptionkey@0"
I20260811 12:52:30.817765 14225 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 12:52:30.821154 14244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:52:30.822508 14225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:52:30.822670 14225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-2/data,/tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-2/wal
uuid: "770cde8817b746d88511047f98019f0a"
format_stamp: "Formatted at 2026-08-11 12:52:30 on dist-test-slave-f0tn"
server_key: "0d2c906f0309f2b86da12e69c8170687"
server_key_iv: "9afffe42270757f1cd6964390cc0d233"
server_key_version: "encryptionkey@0"
I20260811 12:52:30.822863 14225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:52:30.837992 14225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:52:30.841924 14225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:52:30.842175 14225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:52:30.859367 14225 rpc_server.cc:307] RPC server started. Bound to: 127.7.111.124:37447
I20260811 12:52:30.859380 14336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.111.124:37447 every 8 connection(s)
I20260811 12:52:30.860697 14225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-2/data/info.pb
I20260811 12:52:30.864416 14337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:52:30.870625 7613 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu as pid 14225
I20260811 12:52:30.870842 7613 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIqQ7YY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786452375851340-7613-0/minicluster-data/master-2/wal/instance
I20260811 12:52:30.871392 7613 external_mini_cluster.cc:1428] Setting key 2706ba452923d892478b0443e23d2cad
I20260811 12:52:30.871083 14337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40439 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:52:30 dist-test-slave-f0tn krb5kdc[13944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452750, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.124@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
Aug 11 12:52:30 dist-test-slave-f0tn krb5kdc[13944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.124@KRBTEST.COM
I20260811 12:52:30.890931 14343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.873864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34674 (local address 127.7.111.126:40439)
0811 12:52:30.874421 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:30.874427 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:30.875006 (+ 579us) server_negotiation.cc:407] Connection header received
0811 12:52:30.876170 (+ 1164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:30.876175 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:30.876273 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:30.876397 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:30.878040 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.879088 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.880993 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.881521 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.881661 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:30.885271 (+ 3610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:30.885276 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:30.885279 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:30.885311 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:30.887740 (+ 2429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:30.888558 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:30.888562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:30.888564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:30.888684 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:30.889135 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:30.889140 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:30.889142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:30.889500 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:30.889657 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:30.890361 (+ 704us) server_negotiation.cc:299] Negotiation successful
0811 12:52:30.890555 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":303,"threads_started":1}
I20260811 12:52:30.891824 14344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.873927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.126:40439 (local address 127.0.0.1:34674)
0811 12:52:30.874852 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.874891 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 12:52:30.875927 (+ 1036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:52:30.876463 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:52:30.876469 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:52:30.877296 (+ 827us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:52:30.877866 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:30.877891 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.879253 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.879257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:30.880844 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:30.880859 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.881739 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.881744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:30.881912 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:30.882567 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:52:30.882596 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:52:30.885089 (+ 2493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:52:30.887866 (+ 2777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:30.887876 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:30.887887 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:30.888462 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:30.888799 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:30.888804 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:30.888808 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:30.889034 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:30.889615 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:52:30.889621 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:52:30.889983 (+ 362us) client_negotiation.cc:769] Sending connection context
0811 12:52:30.890262 (+ 279us) client_negotiation.cc:240] Negotiation successful
0811 12:52:30.890709 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":485,"threads_started":1}
I20260811 12:52:30.892254 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.873219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52600 (local address 127.7.111.124:37447)
0811 12:52:30.874034 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:30.874046 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:30.874094 (+ 48us) server_negotiation.cc:407] Connection header received
0811 12:52:30.875920 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:30.875941 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:30.876220 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:30.876435 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:30.878195 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.879675 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.881249 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.881994 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.882086 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:30.886022 (+ 3936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:30.886033 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:30.886044 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:30.886073 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:30.888975 (+ 2902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:30.889771 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:30.889778 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:30.889781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:30.889943 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:30.890262 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:30.890267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:30.890270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:30.890666 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:30.890964 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:30.891737 (+ 773us) server_negotiation.cc:299] Negotiation successful
0811 12:52:30.891913 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":370,"threads_started":1}
I20260811 12:52:30.893340 14337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:52:30 dist-test-slave-f0tn krb5kdc[13944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452750, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.124@KRBTEST.COM for kudu/127.7.111.125@KRBTEST.COM
Aug 11 12:52:30 dist-test-slave-f0tn krb5kdc[13944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.126@KRBTEST.COM
I20260811 12:52:30.905972 14085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } attempt: 4
I20260811 12:52:30.906160 14344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.894131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.125:37569 (local address 127.0.0.1:59890)
0811 12:52:30.894288 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.894307 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:52:30.894397 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:52:30.894735 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:52:30.894738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:52:30.895129 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:52:30.895338 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:30.895351 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.896266 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.896270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:30.897579 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:30.897594 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.898294 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.898300 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:30.898489 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:30.898972 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:52:30.898988 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:52:30.900961 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:52:30.903852 (+ 2891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:30.903858 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:30.903862 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:30.904201 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:30.904589 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:30.904594 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:30.904597 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:30.904752 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:30.905329 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:52:30.905332 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:52:30.905504 (+ 172us) client_negotiation.cc:769] Sending connection context
0811 12:52:30.905645 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 12:52:30.905845 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 12:52:30.907032 14220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.894139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59890 (local address 127.7.111.125:37569)
0811 12:52:30.894451 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:30.894458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:30.894476 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:52:30.894533 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:30.894538 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:30.894610 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:30.894713 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:30.895472 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.896133 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.897720 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.898146 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.898269 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:30.901140 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:30.901145 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:30.901149 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:30.901196 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:30.903726 (+ 2530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:30.904322 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:30.904328 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:30.904331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:30.904486 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:30.904854 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:30.904858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:30.904862 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:30.905215 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:30.905415 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:30.905776 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 12:52:30.906034 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184}
I20260811 12:52:30.908051 14337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:52:30.910979 14343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.895149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34686 (local address 127.7.111.126:40439)
0811 12:52:30.895327 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:30.895333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:30.896265 (+ 932us) server_negotiation.cc:407] Connection header received
0811 12:52:30.896310 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:30.896313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:30.896378 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:30.896443 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:30.897866 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.898512 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.899673 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.900067 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.900173 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:30.903919 (+ 3746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:30.903924 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:30.903927 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:30.903957 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:30.907503 (+ 3546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:30.908290 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:30.908295 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:30.908299 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:30.908455 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:30.909279 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:30.909284 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:30.909288 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:30.909812 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:30.910133 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:30.910428 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 12:52:30.910653 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":77}
Aug 11 12:52:30 dist-test-slave-f0tn krb5kdc[13944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452749, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.126@KRBTEST.COM for kudu/127.7.111.124@KRBTEST.COM
Aug 11 12:52:30 dist-test-slave-f0tn krb5kdc[13944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452750, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.124@KRBTEST.COM for kudu/127.7.111.124@KRBTEST.COM
Aug 11 12:52:30 dist-test-slave-f0tn krb5kdc[13944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.111.125@KRBTEST.COM
I20260811 12:52:30.922406 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.906690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:37447 (local address 127.0.0.1:52610)
0811 12:52:30.906865 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.906879 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:52:30.906958 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:52:30.907470 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:52:30.907475 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:52:30.907928 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:52:30.908082 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:30.908090 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.909244 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.909251 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:30.910756 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:30.910772 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.911953 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.911959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:30.912160 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:30.912672 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:52:30.912705 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:52:30.915276 (+ 2571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:52:30.919606 (+ 4330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:30.919612 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:30.919617 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:30.920107 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:30.920514 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:30.920519 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:30.920523 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:30.920694 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:30.921499 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:52:30.921503 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:52:30.921621 (+ 118us) client_negotiation.cc:769] Sending connection context
0811 12:52:30.921772 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 12:52:30.922044 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":20}
I20260811 12:52:30.923422 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.906728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52610 (local address 127.7.111.124:37447)
0811 12:52:30.907111 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:30.907116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:30.907132 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:52:30.907177 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:30.907182 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:30.907255 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:30.907332 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:30.908211 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.909120 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.910931 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.911601 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.911759 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:30.915483 (+ 3724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:30.915491 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:30.915495 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:30.915531 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:30.919453 (+ 3922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:30.920248 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:30.920254 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:30.920258 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:30.920398 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:30.920817 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:30.920822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:30.920826 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:30.921269 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:30.921423 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:30.922738 (+ 1315us) server_negotiation.cc:299] Negotiation successful
0811 12:52:30.922983 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289}
I20260811 12:52:30.923796 14345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.908827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52614 (local address 127.7.111.124:37447)
0811 12:52:30.909822 (+ 995us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:30.909827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:30.909849 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:52:30.909891 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:30.909896 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:30.909957 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:30.910042 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:30.910995 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.912041 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.913674 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.914130 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.914238 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:30.917476 (+ 3238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:30.917484 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:30.917490 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:30.917528 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:30.920422 (+ 2894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:30.921078 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:30.921083 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:30.921086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:30.921213 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:30.921713 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:30.921717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:30.921720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:30.922046 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:30.922187 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:30.922890 (+ 703us) server_negotiation.cc:299] Negotiation successful
0811 12:52:30.923074 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":883,"thread_start_us":708,"threads_started":1}
I20260811 12:52:30.923825 14344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.908808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:37447 (local address 127.0.0.1:52614)
0811 12:52:30.908953 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:30.908970 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:52:30.909064 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:52:30.910239 (+ 1175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:52:30.910241 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:52:30.910653 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:52:30.910824 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:30.910835 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.912437 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.912442 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:30.913560 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:30.913572 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.914405 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.914410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:30.914620 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:30.915157 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:52:30.915180 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:52:30.917238 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:52:30.920536 (+ 3298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:30.920541 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:30.920545 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:30.920960 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:30.921339 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:30.921344 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:30.921348 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:30.921587 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:30.922176 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:52:30.922181 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:52:30.922338 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 12:52:30.922758 (+ 420us) client_negotiation.cc:240] Negotiation successful
0811 12:52:30.923228 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":153}
I20260811 12:52:30.928964 14085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3: Bootstrap starting.
I20260811 12:52:30.929562 14337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 770cde8817b746d88511047f98019f0a: Bootstrap starting.
I20260811 12:52:30.929880 14220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.912986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59906 (local address 127.7.111.125:37569)
0811 12:52:30.913143 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:30.913148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:30.913165 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:52:30.913209 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:30.913213 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:30.913275 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:30.913366 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:30.914735 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.915614 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.917290 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.917793 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.917910 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:30.922768 (+ 4858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:30.922774 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:30.922778 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:30.922813 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:30.926059 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:30.926855 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:30.926860 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:30.926864 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:30.927026 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:30.927496 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:30.927501 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:30.927505 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:30.927850 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:30.928118 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:30.929286 (+ 1168us) server_negotiation.cc:299] Negotiation successful
0811 12:52:30.929556 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:52:30.933022 14085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3: Neither blocks nor log segments found. Creating new log.
I20260811 12:52:30.933022 14337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 770cde8817b746d88511047f98019f0a: Neither blocks nor log segments found. Creating new log.
I20260811 12:52:30.934332 14085 log.cc:824] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3: Log is configured to *not* fsync() on all Append() calls
I20260811 12:52:30.934309 14337 log.cc:824] T 00000000000000000000000000000000 P 770cde8817b746d88511047f98019f0a: Log is configured to *not* fsync() on all Append() calls
I20260811 12:52:30.935105 7613 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:52:30.938740 14085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3: No bootstrap required, opened a new log
I20260811 12:52:30.937626 14040 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "09bd13ff4b654bb8835e2cf73666d1f3"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34686
I20260811 12:52:30.943516 14337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 770cde8817b746d88511047f98019f0a: No bootstrap required, opened a new log
I20260811 12:52:30.945785 14085 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40439 } } peers { permanent_uuid: "54b443bf04034383a9031f8b3c26c25c" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } } peers { permanent_uuid: "770cde8817b746d88511047f98019f0a" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } }
I20260811 12:52:30.946090 14085 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09bd13ff4b654bb8835e2cf73666d1f3, State: Initialized, Role: FOLLOWER
I20260811 12:52:30.946496 14085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40439 } } peers { permanent_uuid: "54b443bf04034383a9031f8b3c26c25c" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } } peers { permanent_uuid: "770cde8817b746d88511047f98019f0a" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } }
I20260811 12:52:30.947623 14349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40439 } } peers { permanent_uuid: "54b443bf04034383a9031f8b3c26c25c" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } } peers { permanent_uuid: "770cde8817b746d88511047f98019f0a" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } } }
I20260811 12:52:30.948215 14349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:52:30.948413 14085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:52:30.948508 14040 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 12:52:30.948693 14040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:52:30.952035 14040 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40439 } } peers { permanent_uuid: "54b443bf04034383a9031f8b3c26c25c" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } } peers { permanent_uuid: "770cde8817b746d88511047f98019f0a" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } }
I20260811 12:52:30.953909 14040 leader_election.cc:288] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3 [CANDIDATE]: Term 1 election: Requested vote from peers 54b443bf04034383a9031f8b3c26c25c (127.7.111.125:37569), 770cde8817b746d88511047f98019f0a (127.7.111.124:37447)
I20260811 12:52:30.953773 14337 raft_consensus.cc:348] T 00000000000000000000000000000000 P 770cde8817b746d88511047f98019f0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40439 } } peers { permanent_uuid: "54b443bf04034383a9031f8b3c26c25c" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } } peers { permanent_uuid: "770cde8817b746d88511047f98019f0a" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } }
I20260811 12:52:30.954921 14337 raft_consensus.cc:729] T 00000000000000000000000000000000 P 770cde8817b746d88511047f98019f0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 770cde8817b746d88511047f98019f0a, State: Initialized, Role: FOLLOWER
I20260811 12:52:30.955444 14337 consensus_queue.cc:260] T 00000000000000000000000000000000 P 770cde8817b746d88511047f98019f0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40439 } } peers { permanent_uuid: "54b443bf04034383a9031f8b3c26c25c" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } } peers { permanent_uuid: "770cde8817b746d88511047f98019f0a" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } }
I20260811 12:52:30.959412 14364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 770cde8817b746d88511047f98019f0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40439 } } peers { permanent_uuid: "54b443bf04034383a9031f8b3c26c25c" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } } peers { permanent_uuid: "770cde8817b746d88511047f98019f0a" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } } }
I20260811 12:52:30.959836 14364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 770cde8817b746d88511047f98019f0a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:52:30.961710 14337 sys_catalog.cc:564] T 00000000000000000000000000000000 P 770cde8817b746d88511047f98019f0a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:52:30.961828 14295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "770cde8817b746d88511047f98019f0a"
I20260811 12:52:30.962373 14295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 770cde8817b746d88511047f98019f0a [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:52:30.966218 14295 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 770cde8817b746d88511047f98019f0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09bd13ff4b654bb8835e2cf73666d1f3 in term 1.
I20260811 12:52:30.967031 14002 leader_election.cc:302] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09bd13ff4b654bb8835e2cf73666d1f3, 770cde8817b746d88511047f98019f0a; no voters:
I20260811 12:52:30.967658 14349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:52:30.968386 14349 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3 [term 1 LEADER]: Becoming Leader. State: Replica: 09bd13ff4b654bb8835e2cf73666d1f3, State: Running, Role: LEADER
I20260811 12:52:30.969048 14349 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40439 } } peers { permanent_uuid: "54b443bf04034383a9031f8b3c26c25c" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } } peers { permanent_uuid: "770cde8817b746d88511047f98019f0a" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } }
I20260811 12:52:30.972935 14371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09bd13ff4b654bb8835e2cf73666d1f3. Latest consensus state: current_term: 1 leader_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40439 } } peers { permanent_uuid: "54b443bf04034383a9031f8b3c26c25c" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } } peers { permanent_uuid: "770cde8817b746d88511047f98019f0a" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } } }
I20260811 12:52:30.973441 14371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3 [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:30.987807 13966 master_runner.cc:186] Error getting master registration for 127.7.111.125:37569: OK, instance_id { permanent_uuid: "54b443bf04034383a9031f8b3c26c25c" instance_seqno: 1786452750315930 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:52:31.002964 13966 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 12:52:31.005882 14386 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:52:31.006012 14386 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:52:31.007140 14381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:52:31.010319 14381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:52:31.011307 14343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.972631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34698 (local address 127.7.111.126:40439)
0811 12:52:30.972821 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:30.972829 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:30.984823 (+ 11994us) server_negotiation.cc:407] Connection header received
0811 12:52:30.985915 (+ 1092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:30.985922 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:30.986010 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:30.986090 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:30.992234 (+ 6144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.993589 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.998397 (+ 4808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.999239 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:30.999370 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:31.003161 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:31.003168 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:31.003172 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:31.003223 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:31.007436 (+ 4213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:31.008407 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:31.008412 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:31.008415 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:31.008597 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:31.009073 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:31.009077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:31.009081 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:31.009494 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:31.010554 (+ 1060us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:31.010698 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 12:52:31.010932 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 12:52:31.011806 14007 tablet.cc:2369] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:52:31.021852 14220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.979704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59912 (local address 127.7.111.125:37569)
0811 12:52:30.979901 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:30.979907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:30.989268 (+ 9361us) server_negotiation.cc:407] Connection header received
0811 12:52:30.989721 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:30.989728 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:30.989812 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:30.989881 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:30.996636 (+ 6755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:30.997880 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:31.000175 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:31.000781 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:31.000875 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:31.007415 (+ 6540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:31.007422 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:31.007427 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:31.007472 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:31.011313 (+ 3841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:31.017104 (+ 5791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:31.017112 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:31.017126 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:31.017308 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:31.018730 (+ 1422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:31.018737 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:31.018740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:31.019140 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:31.019453 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:31.021209 (+ 1756us) server_negotiation.cc:299] Negotiation successful
0811 12:52:31.021484 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 12:52:31.027935 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:30.981476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52630 (local address 127.7.111.124:37447)
0811 12:52:30.981713 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:30.981719 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:31.006513 (+ 24794us) server_negotiation.cc:407] Connection header received
0811 12:52:31.006651 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:31.006657 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:31.006756 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:31.006855 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:31.012925 (+ 6070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:31.014011 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:31.015965 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:31.016760 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:31.016860 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:31.020177 (+ 3317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:31.020184 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:31.020190 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:31.020230 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:31.023602 (+ 3372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:31.024373 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:31.024378 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:31.024381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:31.024529 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:31.024961 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:31.024966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:31.024969 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:31.025366 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:31.025667 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:31.027357 (+ 1690us) server_negotiation.cc:299] Negotiation successful
0811 12:52:31.027586 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:52:31.030494 14295 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 770cde8817b746d88511047f98019f0a [term 1 FOLLOWER]: Refusing update from remote peer 09bd13ff4b654bb8835e2cf73666d1f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:52:31.031477 14371 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "770cde8817b746d88511047f98019f0a" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:52:31.044866 14212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } attempt: 5
I20260811 12:52:31.054226 14364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 770cde8817b746d88511047f98019f0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09bd13ff4b654bb8835e2cf73666d1f3. Latest consensus state: current_term: 1 leader_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40439 } } peers { permanent_uuid: "54b443bf04034383a9031f8b3c26c25c" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } } peers { permanent_uuid: "770cde8817b746d88511047f98019f0a" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } } }
I20260811 12:52:31.054598 14364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 770cde8817b746d88511047f98019f0a [sys.catalog]: This master's current role is: FOLLOWER
Aug 11 12:52:31 dist-test-slave-f0tn krb5kdc[13944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786452750, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.111.125@KRBTEST.COM for kudu/127.7.111.124@KRBTEST.COM
I20260811 12:52:31.066013 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:31.045588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52632 (local address 127.7.111.124:37447)
0811 12:52:31.045770 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:31.045777 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:31.045951 (+ 174us) server_negotiation.cc:407] Connection header received
0811 12:52:31.046157 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:31.046162 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:31.046256 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:31.046354 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:31.047411 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:31.048600 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:31.050532 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:31.051302 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:31.051435 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:31.057700 (+ 6265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:52:31.057707 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:52:31.057712 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:52:31.057753 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:52:31.061954 (+ 4201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:31.062962 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:31.062968 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:31.062972 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:31.063142 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:52:31.063608 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:52:31.063625 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:52:31.063629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:52:31.064026 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:52:31.064173 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:31.065364 (+ 1191us) server_negotiation.cc:299] Negotiation successful
0811 12:52:31.065663 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 12:52:31.066517 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:31.045592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.111.124:37447 (local address 127.0.0.1:52632)
0811 12:52:31.045861 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 12:52:31.045881 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:52:31.046011 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:52:31.046406 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:52:31.046411 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:52:31.046995 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:52:31.047215 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:31.047230 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:31.048764 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:31.048770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:31.050361 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:52:31.050379 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:52:31.053029 (+ 2650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:52:31.053035 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:52:31.053278 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:31.054104 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:52:31.054130 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:52:31.057461 (+ 3331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:52:31.062127 (+ 4666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:31.062132 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:31.062136 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:31.062815 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:31.063281 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:52:31.063287 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:52:31.063290 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:52:31.063469 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:52:31.064239 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:52:31.064244 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:52:31.064412 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 12:52:31.064562 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 12:52:31.064893 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153}
I20260811 12:52:31.073172 14212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54b443bf04034383a9031f8b3c26c25c: Bootstrap starting.
I20260811 12:52:31.074326 14406 mvcc.cc:205] Tried to move back new op lower bound from 7317310468191432704 to 7317310467975213056. Current Snapshot: MvccSnapshot[applied={T|T < 7317310468191432704}]
I20260811 12:52:31.077847 14212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54b443bf04034383a9031f8b3c26c25c: Neither blocks nor log segments found. Creating new log.
I20260811 12:52:31.079291 14212 log.cc:824] T 00000000000000000000000000000000 P 54b443bf04034383a9031f8b3c26c25c: Log is configured to *not* fsync() on all Append() calls
I20260811 12:52:31.080185 14349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40439 } } peers { permanent_uuid: "54b443bf04034383a9031f8b3c26c25c" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } } peers { permanent_uuid: "770cde8817b746d88511047f98019f0a" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } } }
I20260811 12:52:31.080565 14349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3 [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:31.084324 14364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 770cde8817b746d88511047f98019f0a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40439 } } peers { permanent_uuid: "54b443bf04034383a9031f8b3c26c25c" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } } peers { permanent_uuid: "770cde8817b746d88511047f98019f0a" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } } }
I20260811 12:52:31.084609 14364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 770cde8817b746d88511047f98019f0a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:52:31.087040 14381 catalog_manager.cc:1347] Generated new cluster ID: 2b6ab7314f924123890c1116c32006a2
I20260811 12:52:31.087160 14381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:52:31.090498 14212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54b443bf04034383a9031f8b3c26c25c: No bootstrap required, opened a new log
I20260811 12:52:31.102999 14212 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54b443bf04034383a9031f8b3c26c25c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40439 } } peers { permanent_uuid: "54b443bf04034383a9031f8b3c26c25c" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } } peers { permanent_uuid: "770cde8817b746d88511047f98019f0a" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } }
I20260811 12:52:31.103387 14212 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54b443bf04034383a9031f8b3c26c25c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54b443bf04034383a9031f8b3c26c25c, State: Initialized, Role: FOLLOWER
I20260811 12:52:31.103888 14212 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54b443bf04034383a9031f8b3c26c25c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40439 } } peers { permanent_uuid: "54b443bf04034383a9031f8b3c26c25c" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } } peers { permanent_uuid: "770cde8817b746d88511047f98019f0a" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } }
I20260811 12:52:31.109742 14410 catalog_manager.cc:1259] Loaded cluster ID: 2b6ab7314f924123890c1116c32006a2
I20260811 12:52:31.109835 14410 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 770cde8817b746d88511047f98019f0a: loading cluster ID for follower catalog manager: success
W20260811 12:52:31.111150 14410 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 770cde8817b746d88511047f98019f0a: acquiring CA information for follower catalog manager: Not found: root CA entry not found
I20260811 12:52:31.110898 14420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54b443bf04034383a9031f8b3c26c25c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40439 } } peers { permanent_uuid: "54b443bf04034383a9031f8b3c26c25c" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } } peers { permanent_uuid: "770cde8817b746d88511047f98019f0a" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } } }
W20260811 12:52:31.111287 14410 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:52:31.116461 14381 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:52:31.116844 14171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "54b443bf04034383a9031f8b3c26c25c"
I20260811 12:52:31.116767 14212 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54b443bf04034383a9031f8b3c26c25c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:52:31.117333 14171 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54b443bf04034383a9031f8b3c26c25c [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:52:31.099049 14225 master_runner.cc:186] Error getting master registration for 127.7.111.125:37569: OK, instance_id { permanent_uuid: "54b443bf04034383a9031f8b3c26c25c" instance_seqno: 1786452750315930 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:52:31.111313 14420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54b443bf04034383a9031f8b3c26c25c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:52:31.118784 14225 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 12:52:31.120241 14171 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 54b443bf04034383a9031f8b3c26c25c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09bd13ff4b654bb8835e2cf73666d1f3 in term 1.
I20260811 12:52:31.121163 14170 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 54b443bf04034383a9031f8b3c26c25c [term 1 FOLLOWER]: Refusing update from remote peer 09bd13ff4b654bb8835e2cf73666d1f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:52:31.122151 14371 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54b443bf04034383a9031f8b3c26c25c" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:52:31.124009 14381 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:52:31.135949 14381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3: Generated new TSK 0
I20260811 12:52:31.137251 14381 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:52:31.187165 14420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54b443bf04034383a9031f8b3c26c25c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09bd13ff4b654bb8835e2cf73666d1f3. Latest consensus state: current_term: 1 leader_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40439 } } peers { permanent_uuid: "54b443bf04034383a9031f8b3c26c25c" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } } peers { permanent_uuid: "770cde8817b746d88511047f98019f0a" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } } }
I20260811 12:52:31.187296 14371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40439 } } peers { permanent_uuid: "54b443bf04034383a9031f8b3c26c25c" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } } peers { permanent_uuid: "770cde8817b746d88511047f98019f0a" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } } }
I20260811 12:52:31.187528 14371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09bd13ff4b654bb8835e2cf73666d1f3 [sys.catalog]: This master's current role is: LEADER
I20260811 12:52:31.187530 14420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54b443bf04034383a9031f8b3c26c25c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:52:31.187731 14429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54b443bf04034383a9031f8b3c26c25c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09bd13ff4b654bb8835e2cf73666d1f3" member_type: VOTER last_known_addr { host: "127.7.111.126" port: 40439 } } peers { permanent_uuid: "54b443bf04034383a9031f8b3c26c25c" member_type: VOTER last_known_addr { host: "127.7.111.125" port: 37569 } } peers { permanent_uuid: "770cde8817b746d88511047f98019f0a" member_type: VOTER last_known_addr { host: "127.7.111.124" port: 37447 } } }
I20260811 12:52:31.188186 14429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54b443bf04034383a9031f8b3c26c25c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:52:31.216569 14451 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260811 12:52:31.255607 14220 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:31.247362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59914 (local address 127.7.111.125:37569)
0811 12:52:31.247562 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:31.247569 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:31.253361 (+ 5792us) server_negotiation.cc:407] Connection header received
0811 12:52:31.253728 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:31.253735 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:31.253865 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:31.253943 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:31.255352 (+ 1409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59914: BlockingRecv error: recv got EOF from 127.0.0.1:59914 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 12:52:31.256168 14343 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:31.246925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34714 (local address 127.7.111.126:40439)
0811 12:52:31.247167 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:31.247174 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:31.253756 (+ 6582us) server_negotiation.cc:407] Connection header received
0811 12:52:31.254002 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:31.254008 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:31.254091 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:31.254162 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:31.255917 (+ 1755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34714: BlockingRecv error: recv got EOF from 127.0.0.1:34714 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260811 12:52:31.258424 14342 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:31.248473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52636 (local address 127.7.111.124:37447)
0811 12:52:31.248642 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:31.248648 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:31.255823 (+ 7175us) server_negotiation.cc:407] Connection header received
0811 12:52:31.256861 (+ 1038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:31.256868 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:31.256958 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:31.257089 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:31.258126 (+ 1037us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52636: BlockingRecv error: recv got EOF from 127.0.0.1:52636 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 12:52:31.265507 14220 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:31.263299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59930 (local address 127.7.111.125:37569)
0811 12:52:31.263496 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:31.263502 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:31.263992 (+ 490us) server_negotiation.cc:407] Connection header received
0811 12:52:31.264199 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:31.264205 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:31.264334 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:31.264445 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:31.265216 (+ 771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59930: BlockingRecv error: recv got EOF from 127.0.0.1:59930 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260811 12:52:31.271938 14220 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:31.270395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59938 (local address 127.7.111.125:37569)
0811 12:52:31.270560 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:31.270565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:31.270584 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:52:31.270666 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:31.270671 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:31.270757 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:31.270863 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:31.271718 (+ 855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59938: BlockingRecv error: recv got EOF from 127.0.0.1:59938 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 12:52:31.283617 14220 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:31.279395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59952 (local address 127.7.111.125:37569)
0811 12:52:31.279568 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:31.279573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:31.282033 (+ 2460us) server_negotiation.cc:407] Connection header received
0811 12:52:31.282248 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:31.282254 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:31.282367 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:31.282479 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:31.283357 (+ 878us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59952: BlockingRecv error: recv got EOF from 127.0.0.1:59952 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 12:52:31.293299 14220 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:31.291077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59956 (local address 127.7.111.125:37569)
0811 12:52:31.291312 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:31.291319 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:31.291823 (+ 504us) server_negotiation.cc:407] Connection header received
0811 12:52:31.292013 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:31.292020 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:31.292113 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:31.292220 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:31.293065 (+ 845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59956: BlockingRecv error: recv got EOF from 127.0.0.1:59956 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
W20260811 12:52:31.307435 14220 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:31.304963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59968 (local address 127.7.111.125:37569)
0811 12:52:31.305166 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:31.305173 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:31.305631 (+ 458us) server_negotiation.cc:407] Connection header received
0811 12:52:31.305961 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:31.305967 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:31.306071 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:31.306146 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:31.307198 (+ 1052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59968: BlockingRecv error: recv got EOF from 127.0.0.1:59968 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 12:52:31.355521 14220 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:31.353096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59970 (local address 127.7.111.125:37569)
0811 12:52:31.353292 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:31.353298 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:31.353915 (+ 617us) server_negotiation.cc:407] Connection header received
0811 12:52:31.354115 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:31.354121 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:31.354242 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:31.354358 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:31.355283 (+ 925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59970: BlockingRecv error: recv got EOF from 127.0.0.1:59970 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 12:52:31.394294 14220 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:31.391797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59972 (local address 127.7.111.125:37569)
0811 12:52:31.391999 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:31.392006 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:31.392600 (+ 594us) server_negotiation.cc:407] Connection header received
0811 12:52:31.392826 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:31.392832 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:31.392932 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:31.393074 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:31.394008 (+ 934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59972: BlockingRecv error: recv got EOF from 127.0.0.1:59972 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 12:52:31.466229 14220 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:31.463733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59984 (local address 127.7.111.125:37569)
0811 12:52:31.463961 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:31.463969 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:31.464506 (+ 537us) server_negotiation.cc:407] Connection header received
0811 12:52:31.464732 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:31.464738 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:31.464847 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:31.465007 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:31.465997 (+ 990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59984: BlockingRecv error: recv got EOF from 127.0.0.1:59984 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260811 12:52:31.599925 14220 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:31.598088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59990 (local address 127.7.111.125:37569)
0811 12:52:31.598273 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:31.598278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:31.598650 (+ 372us) server_negotiation.cc:407] Connection header received
0811 12:52:31.598842 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:31.598846 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:31.598917 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:31.598994 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:31.599686 (+ 692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59990: BlockingRecv error: recv got EOF from 127.0.0.1:59990 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 12:52:31.863030 14220 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:52:31.860884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60000 (local address 127.7.111.125:37569)
0811 12:52:31.861116 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:31.861123 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:31.861529 (+ 406us) server_negotiation.cc:407] Connection header received
0811 12:52:31.861731 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:31.861735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:31.861824 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:31.861909 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:52:31.862816 (+ 907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60000: BlockingRecv error: recv got EOF from 127.0.0.1:60000 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 12:52:32.114683 14410 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 770cde8817b746d88511047f98019f0a: acquiring CA information for follower catalog manager: success
I20260811 12:52:32.116518 14410 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 770cde8817b746d88511047f98019f0a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:52:32.222713 14451 catalog_manager.cc:1259] Loaded cluster ID: 2b6ab7314f924123890c1116c32006a2
I20260811 12:52:32.222879 14451 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 54b443bf04034383a9031f8b3c26c25c: loading cluster ID for follower catalog manager: success
I20260811 12:52:32.227144 14451 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 54b443bf04034383a9031f8b3c26c25c: acquiring CA information for follower catalog manager: success
I20260811 12:52:32.230051 14451 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 54b443bf04034383a9031f8b3c26c25c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:52:32.386623 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:32.378616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60016 (local address 127.7.111.125:37569)
0811 12:52:32.380634 (+ 2018us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:32.380639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:32.380658 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:52:32.380729 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:32.380733 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:32.380891 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:32.381013 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:52:32.381538 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:32.382408 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:32.384318 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:32.384788 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:32.384880 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:32.385457 (+ 577us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:52:32.386009 (+ 552us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:52:32.386124 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:32.386252 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 12:52:32.386321 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1884,"thread_start_us":1743,"threads_started":1}
I20260811 12:52:32.414904 14577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:32.394335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52640 (local address 127.7.111.124:37447)
0811 12:52:32.408698 (+ 14363us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:32.408703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:32.408726 (+ 23us) server_negotiation.cc:407] Connection header received
0811 12:52:32.408830 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:32.408834 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:32.408998 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:32.409164 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:52:32.409820 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:32.410555 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:32.412471 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:32.412873 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:32.413019 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:32.413729 (+ 710us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:52:32.414229 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:52:32.414338 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:32.414520 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 12:52:32.414596 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14230,"thread_start_us":14921,"threads_started":1}
I20260811 12:52:32.431102 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:32.421476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60028 (local address 127.7.111.125:37569)
0811 12:52:32.421651 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:32.421656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:32.423915 (+ 2259us) server_negotiation.cc:407] Connection header received
0811 12:52:32.424202 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:32.424207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:32.424297 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:32.424361 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:52:32.425168 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:32.426418 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:32.428737 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:32.429423 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:32.429514 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:32.430210 (+ 696us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:52:32.430350 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:52:32.430428 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:32.430671 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 12:52:32.430756 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 12:52:32.431825 14586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:32.421125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34722 (local address 127.7.111.126:40439)
0811 12:52:32.423250 (+ 2125us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:32.423256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:32.423839 (+ 583us) server_negotiation.cc:407] Connection header received
0811 12:52:32.423971 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:32.423976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:32.424491 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:32.424676 (+ 185us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:52:32.425453 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:32.426497 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:32.428733 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:32.429360 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:32.429515 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:32.430250 (+ 735us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:52:32.430735 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:52:32.430822 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:32.431448 (+ 626us) server_negotiation.cc:299] Negotiation successful
0811 12:52:32.431534 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2007,"thread_start_us":1881,"threads_started":1}
I20260811 12:52:32.436511 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 13966
I20260811 12:52:32.437482 14076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:52:32.438750 14577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:52:32.424143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52654 (local address 127.7.111.124:37447)
0811 12:52:32.424393 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 12:52:32.424398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:52:32.431361 (+ 6963us) server_negotiation.cc:407] Connection header received
0811 12:52:32.432992 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:52:32.432998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:52:32.433114 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:52:32.433165 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:52:32.433717 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:32.434776 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:32.436575 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:52:32.437197 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:52:32.437327 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:52:32.437893 (+ 566us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:52:32.438043 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:52:32.438104 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 12:52:32.438327 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 12:52:32.438424 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260811 12:52:32.554092 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 13966
I20260811 12:52:32.589926 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 14094
I20260811 12:52:32.591063 14205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:52:32.698246 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 14094
I20260811 12:52:32.728660 7613 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu pid 14225
I20260811 12:52:32.729806 14331 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:52:32.839206 7613 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIqQ7YY/build/asan/bin/kudu with pid 14225
2026-08-11T12:52:32Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5154 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10380 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (376944 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST